-- Logs begin at Wed 2022-08-24 04:05:35 UTC, end at Wed 2022-08-24 05:28:02 UTC. -- Aug 24 04:40:10.090182 dv-cinder-854265-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 04:40:10.099243 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 04:40:10 2022] *** Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: nodename: dv-cinder-854265-1-iscsi Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: machine: x86_64 Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: clock source: unix Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: pcre jit disabled Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: detected number of CPU cores: 4 Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: current working directory: / Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: detected binary path: /usr/bin/uwsgi-core Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: your processes number limit is 47680 Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: your memory page size is 4096 bytes Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: detected max file descriptor number: 2048 Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: lock engine: pthread robust mutexes Aug 24 04:40:10.101616 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: thunder lock: enabled Aug 24 04:40:10.102585 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 04:40:10.102585 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 04:40:10.142222 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: Python main interpreter initialized at 0x5651821109c0 Aug 24 04:40:10.142222 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: python threads support enabled Aug 24 04:40:10.142222 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: your server socket listen backlog is limited to 100 connections Aug 24 04:40:10.142222 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: your mercy for graceful operations on workers is 90 seconds Aug 24 04:40:10.143114 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 04:40:10.143114 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: *** Operational MODE: preforking *** Aug 24 04:40:10.143114 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: *** uWSGI is running in multiple interpreter mode *** Aug 24 04:40:10.143114 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: spawned uWSGI master process (pid: 82559) Aug 24 04:40:10.143191 dv-cinder-854265-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 04:40:10.143842 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: spawned uWSGI worker 1 (pid: 82574, cores: 1) Aug 24 04:40:10.144722 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: spawned uWSGI worker 2 (pid: 82575, cores: 1) Aug 24 04:40:10.144722 dv-cinder-854265-1-iscsi devstack@keystone.service[82559]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 04:40:14.524643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 04:40:14.524643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: from cryptography.utils import int_from_bytes Aug 24 04:40:14.525401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 04:40:14.525401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: from cryptography.utils import int_from_bytes Aug 24 04:40:14.525816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 04:40:14.525816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: from cryptography.utils import int_from_bytes Aug 24 04:40:14.525919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 04:40:14.525919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: from cryptography.utils import int_from_bytes Aug 24 04:40:14.575516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 04:40:14.575709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 04:40:14.575885 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] command line args: [] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 04:40:14.576179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 04:40:14.576361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ================================================================================ {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 04:40:14.576615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] admin_token = **** {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.576861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] config_dir = [] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.577048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.577881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] config_source = [] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.578077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] debug = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.578220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.578756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.578756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.578993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.579268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.579504 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] list_limit = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.579685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] log_config_append = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.579878 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.580062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] log_dir = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.580238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] log_file = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.580476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.580727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.580973 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.581220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.581406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.581579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.581799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.582044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.582278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.582511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.582744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.582977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.583250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.583497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.583711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 04:40:14.583803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.583908 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 04:40:14.584102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] command line args: [] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 04:40:14.584252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.105/identity {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.584369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 04:40:14.584455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] publish_errors = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.584596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ================================================================================ {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 04:40:14.584680 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.584873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] admin_token = **** {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.584970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.585077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.585156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] config_dir = [] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.585304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.585452 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.585545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] config_source = [] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.585668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.585757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] debug = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.585833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] standard_threads = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.585906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] insecure_debug = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] use_journal = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] use_json = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586843 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] use_stderr = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] list_limit = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.586987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] use_syslog = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.587083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] log_config_append = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.587227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.587314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.587389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] log_dir = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.587503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.587589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] log_file = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.587745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.587837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.588021 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.588207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.588330 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.588453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.588576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.588743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.588863 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.588998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.589124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.589255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.589379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.589506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.589638 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.589757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.589873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.590000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.590120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.590236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.590351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.590467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.590589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.590788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.590973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.591116 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.591245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] notification_format = cadf {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.591363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.591485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.591747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.591747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.591975 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.105/identity {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.591975 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] publish_errors = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.592242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.592344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.592449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.592552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.592655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.592761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.592904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.593536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.593645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.593749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.593847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.593963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] standard_threads = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.594075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.594175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] strict_password_check = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.594310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.594427 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.594526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.594630 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] use_eventlog = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.594630 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] use_journal = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.594817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.594914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] use_json = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.595010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.598834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] use_stderr = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.598834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] use_syslog = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.599172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.599248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] watch_log_file = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 04:40:14.599321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.599404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.599484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.599566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.599646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.599646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.599782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.599782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.599917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.599997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600149 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.600949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601098 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601393 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601539 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.601902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602696 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.602915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.603012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.603134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.603231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.603340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.603439 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.603536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.603632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.603741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.603893 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.603987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.604067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.604186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.604302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.604451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.604576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.604666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.604748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.604822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.604940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605270 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605852 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.605982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.606063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.606136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.606286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.606368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.606487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.606574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.606649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.606774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.606853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.606933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.607993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608270 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.608965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.609929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.610008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.610157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.610239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.610397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.610486 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.610574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.610651 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.610724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.610808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.610895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611650 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611875 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.611957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.612032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.612105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.612178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.612249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.612321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.612432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.612516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] auth.external = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.612683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.enabled = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.612764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.612925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.613006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.613165 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.613253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.613409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.613490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] auth.password = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.613647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.613728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] auth.token = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.613885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.613968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.614123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.614249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.614626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.614746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.614876 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.614995 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.615146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.615267 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.615385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.615509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.615634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.615751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.615867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] credential.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.615993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.616155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.616319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.616442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.616543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.616664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.616788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.616907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.617054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.617054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.617287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.617420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.617539 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.617656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.617783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.617906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.618024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.618150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.618269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.618389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.618506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.618623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.618755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.618875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.618992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.619167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.619289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.619417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.619544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.619664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.619782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.619899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.620016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] federation.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.620134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] auth.external = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.620260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.620379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] auth.mapped = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.620498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.620616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.620734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.620852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.620981 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.621100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] auth.password = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.621218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.621341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] auth.token = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.621460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.621579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.621705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.621824 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] catalog.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.621948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.622067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.622184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.622302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.622428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.622547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.622665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.622783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.622909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.623040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.623040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] credential.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.623348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.623467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] credential.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.623590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.623710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.623827 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.623827 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.624059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.624059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.624407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.624511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.624610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.624721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.624829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.624948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.625066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.625183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.625301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.625301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.625532 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.625650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.625769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.625886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.625886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.626122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.626242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.626359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.626477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.626595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.626712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.626840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.626960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.627094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.627094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] federation.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.627320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.627438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.627569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.627689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] federation.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.627808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.627926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.628043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.628173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.628293 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.628411 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.628529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.628646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.628764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.628764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.628995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.629114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.629232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.629350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.629467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.629593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.629713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.629830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.629948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.630065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.630183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.630308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.630428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.630551 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.630670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.630787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.630911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.631088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.631221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.631349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.631469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.631586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.631704 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.631830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.631951 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.632070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.632187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.632187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.632406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.632532 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.632652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.632769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.632887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.633004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.633121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.633252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.633372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.633490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.633608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.633725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.633842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.633968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.634089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.634207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.634325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.634443 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.634560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.634685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.634805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.634923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.635055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.635176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.635295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.635425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.635545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.635669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.635789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.635906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.636024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.636150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.636271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.636396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.636516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.636633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.636751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.636877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.636997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.637114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.637238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.637355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.637472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.637599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.637752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.637870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.637988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.638105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.638222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.638349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.638469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.638586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.638704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.638821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.password = **** {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.638938 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.639122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.639244 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.639363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.639480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.639597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.639714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.640322 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.640431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.640548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.640650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.640748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.640847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.640971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.641091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.641208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.641325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.641325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.641546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.641672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.641792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.641934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.642053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.642170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.642296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.642415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.642534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.642652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.642775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.642894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.643020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.643159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.643278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.643394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.643511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.643629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.643761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.643882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.643999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.644116 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.644232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.644349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.644474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.644595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] resource.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.644713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.644830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.644946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.645063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.645188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.645308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.645425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.645542 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.645660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.645777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.645902 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.646022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.646139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.646262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.646380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.646498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.646690 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.646848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.646970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.647155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] role.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.647275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.647392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] role.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.647523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.647669 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.647791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.647909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.648026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.648151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.648279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.648399 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.648517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.648634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.648751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.648869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.648996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.649115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.649233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.649350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.649468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.649585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.649711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] policy.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.649832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.649949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.650066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.650184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.650374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.650488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.650590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.650689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] receipt.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.650788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.650887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.650887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.656235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.656355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.656457 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.656557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.656640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.656709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.656801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.656878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] resource.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.656953 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] resource.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] revoke.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.657931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658005 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] role.cache_time = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] role.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] role.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] role.list_limit = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.658934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] token.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659836 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.659991 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660331 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.660926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661452 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661602 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.661954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.connection = **** {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662621 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.662998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] token.cache_time = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] token.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.663970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] token.provider = fernet {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664244 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664244 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82574) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 04:40:14.664845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] trust.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.664845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.665159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.665232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.665431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.665663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.665907 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.666198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.666461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.connection = **** {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.666711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.666960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.667224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.667469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.667813 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.668060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.668301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.668541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.668790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.669030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.669277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.669514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.669761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.669999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.670246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.670477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.670718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.670953 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 04:40:14.671336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=82575) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 04:40:14.692967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.693198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.693498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.696130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.696365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.696588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.701901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.705824 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.731776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.732005 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.732275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.732506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.793219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.793410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.842563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.842796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.843192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:14.843520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.008425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.008658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.008862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.009051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.009155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.009284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.009369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.009498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.009585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.009713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.009929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.010142 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.042060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.042262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.042355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.042499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.042587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.042722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.042804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.042943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.043056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.043187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.043311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.043395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.043470 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.043605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.062041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.063101 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.075340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.075458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.075576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.075679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.075804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.075888 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.076030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.076113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.076252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.076334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.076454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.076537 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.076664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82575) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.076787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=82574) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 04:40:15.094596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.094809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 04:40:15.095037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.095180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.095296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 04:40:15.102024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 04:40:15.102120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.102199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.102275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 04:40:15.102351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.102491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.102591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.102672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.102672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.102808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.102883 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.106058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.106337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.106472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.106610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.106763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.106898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.106996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.107312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.114642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.115077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.117868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.118154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.118262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.118440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.118549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.118726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.118820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.119013 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.119126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.119302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.119408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.119580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.119669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.119905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.120001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.120200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.120282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.120562 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.120647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.120843 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.120926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.121113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.144595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.144698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.146070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.146155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.146354 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.147934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.151087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.151179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.151372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.151455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.151654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.151801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.158183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.158286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.158467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.158552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.162587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.162685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.162875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.162963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.163182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.163304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.163465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.163551 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.163740 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.163824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.164019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.164119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.164291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.164375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.164561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.164645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.182695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.182822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.182989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.183104 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.186915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.187104 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.190873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.190964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.191071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.191222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.195220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.195485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.198375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.198642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.201622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.201921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.202011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.202206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.202297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.202500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.202594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.202787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.202880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.203095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.203183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.203424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.219127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.219423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.219688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.219787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.219950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.220037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.220298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.220565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.226850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.227560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.228403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.228680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.229096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.229377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.231359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.231647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.231941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.232089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.232232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.232379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.232508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.232676 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.232970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.233250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.244706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.244990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.245270 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.245550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.245843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.245949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.246150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.246236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.246429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.246502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.246719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.247007 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.247310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.247590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.262633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.263758 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.265670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.266847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.268936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.269780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.270385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.271270 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.271901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.272184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.272478 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.272764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.273052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.273349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.278993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.279930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.280472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.281396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.283535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.283828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.284120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.284405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.284487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.284756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.285049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.285340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.292446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.292735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.293017 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.293416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.293667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.293956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.300818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82575) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.301104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.301389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.301472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=82574) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 04:40:15.301670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.301753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.301946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.302038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.302236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.302320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.302509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.302593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.302779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.302865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.303093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.303181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.303374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.303455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.303644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.303727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.303918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.304000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.304191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.304281 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.304451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82575) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.304533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.304728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.304991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=82574) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 04:40:15.339700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5651821109c0 pid: 82574 (default app) Aug 24 04:40:15.340052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5651821109c0 pid: 82575 (default app) Aug 24 04:40:15.343142 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69781b-7834-4c5f-9e56-c4441dada6c9 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:15.343433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69781b-7834-4c5f-9e56-c4441dada6c9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:15.343706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e69781b-7834-4c5f-9e56-c4441dada6c9 None None] PATH_INFO: `/v3` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:15.347940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1/1] 10.222.0.105 () {50 vars in 796 bytes} [Wed Aug 24 04:40:15 2022] GET /identity/v3/ => generated 256 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:22.840669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711f694-4623-4d01-aafe-c73558d2e97a None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:22.841179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711f694-4623-4d01-aafe-c73558d2e97a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:22.841268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0711f694-4623-4d01-aafe-c73558d2e97a None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:22.845512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1/2] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:22 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:40:22.852119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:22.852542 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:22.852665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:22.922354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: INFO dbcounter [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] Registered counter for database keystone Aug 24 04:40:22.941471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG oslo_db.sqlalchemy.engines [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82574) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 04:40:22.947581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writer thread running {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 04:40:23.125820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG passlib.handlers.bcrypt [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=82574) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 04:40:23.126175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG passlib.handlers.bcrypt [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82574) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 04:40:23.180914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:23.182508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:23.310525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 2/3] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:22 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 04:40:23.763775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711f694-4623-4d01-aafe-c73558d2e97a None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:23.832331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: INFO dbcounter [None req-0711f694-4623-4d01-aafe-c73558d2e97a None None] Registered counter for database keystone Aug 24 04:40:23.853937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG oslo_db.sqlalchemy.engines [None req-0711f694-4623-4d01-aafe-c73558d2e97a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=82575) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 04:40:23.860500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writer thread running {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 04:40:23.979891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0711f694-4623-4d01-aafe-c73558d2e97a None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:23.984183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7425ee-18dd-4f94-abb7-13166c9028d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:23.984980 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7425ee-18dd-4f94-abb7-13166c9028d9 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:23.985406 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7425ee-18dd-4f94-abb7-13166c9028d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:23.985684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7425ee-18dd-4f94-abb7-13166c9028d9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:23.993731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7425ee-18dd-4f94-abb7-13166c9028d9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:24.048793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9a7425ee-18dd-4f94-abb7-13166c9028d9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 04:40:24.048793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7425ee-18dd-4f94-abb7-13166c9028d9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:24.055888 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 2/4] 10.222.0.105 () {58 vars in 1329 bytes} [Wed Aug 24 04:40:23 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 04:40:24.061934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:24.077327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcfc15-d859-4821-9934-0c2815a05da1 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:24.081861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ce32c-aa8f-4126-853b-57a892f0bc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:24.082973 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ce32c-aa8f-4126-853b-57a892f0bc7d admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:24.083293 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ce32c-aa8f-4126-853b-57a892f0bc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:24.083568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ce32c-aa8f-4126-853b-57a892f0bc7d admin admin] PATH_INFO: `/v3/projects/e02e910be5274b36b8f63c2185a111d4` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:24.085163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ce32c-aa8f-4126-853b-57a892f0bc7d admin admin] RBAC: Authorizing `identity:get_project(project_id=e02e910be5274b36b8f63c2185a111d4)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:24.138619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: WARNING keystone.common.rbac_enforcer.enforcer [None req-560ce32c-aa8f-4126-853b-57a892f0bc7d admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 04:40:24.138957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ce32c-aa8f-4126-853b-57a892f0bc7d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:24.144230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 3/5] 10.222.0.105 () {56 vars in 1212 bytes} [Wed Aug 24 04:40:24 2022] GET /identity/v3/projects/e02e910be5274b36b8f63c2185a111d4 => generated 345 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:26.152541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce960e-5510-4636-86b6-387f1dffb606 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:26.153084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce960e-5510-4636-86b6-387f1dffb606 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:26.153084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce960e-5510-4636-86b6-387f1dffb606 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:26.157167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 3/6] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:26 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:26.162865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfdd4-9da9-4af5-9242-fbe2e149960a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:26.163312 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfdd4-9da9-4af5-9242-fbe2e149960a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:26.163467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386cfdd4-9da9-4af5-9242-fbe2e149960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:26.183788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-386cfdd4-9da9-4af5-9242-fbe2e149960a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:26.185335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-386cfdd4-9da9-4af5-9242-fbe2e149960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:26.194076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 4/7] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:26 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 04:40:26.460692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce960e-5510-4636-86b6-387f1dffb606 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:26.473757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce960e-5510-4636-86b6-387f1dffb606 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:26.478054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6410a-1054-4ef2-88a4-438f9b083344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:26.478649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6410a-1054-4ef2-88a4-438f9b083344 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:26.478930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6410a-1054-4ef2-88a4-438f9b083344 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:26.479232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6410a-1054-4ef2-88a4-438f9b083344 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:26.486812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6410a-1054-4ef2-88a4-438f9b083344 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:26.488623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6410a-1054-4ef2-88a4-438f9b083344 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:26.497958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 4/8] 10.222.0.105 () {58 vars in 1329 bytes} [Wed Aug 24 04:40:26 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 04:40:26.503367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cfdd4-9da9-4af5-9242-fbe2e149960a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:26.513201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386cfdd4-9da9-4af5-9242-fbe2e149960a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:26.517552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c5eca-efe0-4c6b-97fa-6ab6ca060628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:26.518171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c5eca-efe0-4c6b-97fa-6ab6ca060628 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:26.518459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c5eca-efe0-4c6b-97fa-6ab6ca060628 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:26.518732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c5eca-efe0-4c6b-97fa-6ab6ca060628 admin admin] PATH_INFO: `/v3/users/f6a4ad480e8041b39ccfa602ac02988c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:26.520648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c5eca-efe0-4c6b-97fa-6ab6ca060628 admin admin] RBAC: Authorizing `identity:get_user(user_id=f6a4ad480e8041b39ccfa602ac02988c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:26.522469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c5eca-efe0-4c6b-97fa-6ab6ca060628 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:26.524009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 5/9] 10.222.0.105 () {56 vars in 1203 bytes} [Wed Aug 24 04:40:26 2022] GET /identity/v3/users/f6a4ad480e8041b39ccfa602ac02988c => generated 248 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:28.483847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f0c51-f5a3-440b-a0e4-a85078a467b8 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:28.484337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f0c51-f5a3-440b-a0e4-a85078a467b8 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:28.484337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f0c51-f5a3-440b-a0e4-a85078a467b8 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:28.485624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 5/10] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:28.490800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca492ca-68bb-44a6-bdc6-04423b4667fe None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:28.491271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca492ca-68bb-44a6-bdc6-04423b4667fe None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:28.491390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca492ca-68bb-44a6-bdc6-04423b4667fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:28.510380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-bca492ca-68bb-44a6-bdc6-04423b4667fe None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:28.511595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-bca492ca-68bb-44a6-bdc6-04423b4667fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:28.537085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8fba6-62da-499d-99d6-a8187ed30999 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:28.537367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8fba6-62da-499d-99d6-a8187ed30999 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:28.537641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8fba6-62da-499d-99d6-a8187ed30999 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:28.539249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 6/11] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:40:28.544433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09f9d7-175f-4637-8e59-d6c1bf15761b None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:28.544713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09f9d7-175f-4637-8e59-d6c1bf15761b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:28.544984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09f9d7-175f-4637-8e59-d6c1bf15761b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:28.587143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG passlib.handlers.bcrypt [None req-6f09f9d7-175f-4637-8e59-d6c1bf15761b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=82575) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 04:40:28.587557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG passlib.handlers.bcrypt [None req-6f09f9d7-175f-4637-8e59-d6c1bf15761b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=82575) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 04:40:28.590751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 6/12] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:28.633727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-6f09f9d7-175f-4637-8e59-d6c1bf15761b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:28.635350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6f09f9d7-175f-4637-8e59-d6c1bf15761b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:28.733195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 7/13] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:40:28.855437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca492ca-68bb-44a6-bdc6-04423b4667fe None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:28.879437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:28.958868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca492ca-68bb-44a6-bdc6-04423b4667fe None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:28.992234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89cbc4-d279-4561-9d57-260ceb47188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:28.993440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89cbc4-d279-4561-9d57-260ceb47188a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:28.993720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89cbc4-d279-4561-9d57-260ceb47188a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:28.993993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89cbc4-d279-4561-9d57-260ceb47188a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:29.002962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09f9d7-175f-4637-8e59-d6c1bf15761b None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:29.005502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89cbc4-d279-4561-9d57-260ceb47188a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-5b89cbc4-d279-4561-9d57-260ceb47188a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:29.014407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:29.015988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:29.016802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 7/14] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:40:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:29.021454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89cbc4-d279-4561-9d57-260ceb47188a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:29.103550 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:29.107941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f09f9d7-175f-4637-8e59-d6c1bf15761b None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:29.110191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:29.121332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89cbc4-d279-4561-9d57-260ceb47188a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:29.145296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d7f0d-5ba5-418b-a11b-2653d40f14e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:29.146158 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7f0d-5ba5-418b-a11b-2653d40f14e0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:29.146439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7f0d-5ba5-418b-a11b-2653d40f14e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:29.146713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7f0d-5ba5-418b-a11b-2653d40f14e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:29.154609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3b36b-fdc1-49ae-9835-58f1fff618f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:29.155243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3b36b-fdc1-49ae-9835-58f1fff618f9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:29.155520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3b36b-fdc1-49ae-9835-58f1fff618f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:29.155793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3b36b-fdc1-49ae-9835-58f1fff618f9 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:29.156443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3b36b-fdc1-49ae-9835-58f1fff618f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:29.156735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3b36b-fdc1-49ae-9835-58f1fff618f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:29.158110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3b36b-fdc1-49ae-9835-58f1fff618f9 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:29.170164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 8/15] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:29 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:29.175765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3b36b-fdc1-49ae-9835-58f1fff618f9 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:29.216599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d7f0d-5ba5-418b-a11b-2653d40f14e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:29.218122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d7f0d-5ba5-418b-a11b-2653d40f14e0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:29.276946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3b36b-fdc1-49ae-9835-58f1fff618f9 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:29.310262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4af60-1abd-4f15-b72c-d065efbb6982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:29.310885 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4af60-1abd-4f15-b72c-d065efbb6982 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:29.311197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4af60-1abd-4f15-b72c-d065efbb6982 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:29.311475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4af60-1abd-4f15-b72c-d065efbb6982 None admin] PATH_INFO: `/v3/users/f6a4ad480e8041b39ccfa602ac02988c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:29.312604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4af60-1abd-4f15-b72c-d065efbb6982 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a4ad480e8041b39ccfa602ac02988c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:29.314102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4af60-1abd-4f15-b72c-d065efbb6982 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:29.314221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 8/16] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:29 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 04:40:29.316042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 9/17] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:29 2022] GET /identity/v3/users/f6a4ad480e8041b39ccfa602ac02988c => generated 248 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:29.319722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d7f0d-5ba5-418b-a11b-2653d40f14e0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:29.321770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4af60-1abd-4f15-b72c-d065efbb6982 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:29.322500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:29.336411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:29.417455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d7f0d-5ba5-418b-a11b-2653d40f14e0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:29.422398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4af60-1abd-4f15-b72c-d065efbb6982 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:29.451987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd56a24-d82b-4731-b12e-e63e54b3cc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:29.452835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd56a24-d82b-4731-b12e-e63e54b3cc79 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:29.453120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd56a24-d82b-4731-b12e-e63e54b3cc79 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:29.453389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd56a24-d82b-4731-b12e-e63e54b3cc79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:29.457203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e93c66-1c89-4ebc-99e8-de3febb51c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:29.457825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e93c66-1c89-4ebc-99e8-de3febb51c52 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:29.458103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e93c66-1c89-4ebc-99e8-de3febb51c52 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:29.458381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e93c66-1c89-4ebc-99e8-de3febb51c52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:29.459517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e93c66-1c89-4ebc-99e8-de3febb51c52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:29.460988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e93c66-1c89-4ebc-99e8-de3febb51c52 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:29.463430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 10/18] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:29 2022] GET /identity/v3/domains/default => generated 200 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:29.469126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e93c66-1c89-4ebc-99e8-de3febb51c52 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:29.472424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd56a24-d82b-4731-b12e-e63e54b3cc79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:29.473893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd56a24-d82b-4731-b12e-e63e54b3cc79 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-1bd56a24-d82b-4731-b12e-e63e54b3cc79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:40:29.492439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:29.494647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:40:29.495819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:40:29.495819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:40:29.495819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:29.495819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 9/19] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:40:29.499773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd56a24-d82b-4731-b12e-e63e54b3cc79 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:29.551000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:29.555399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:29.570157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e93c66-1c89-4ebc-99e8-de3febb51c52 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:29.598332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd56a24-d82b-4731-b12e-e63e54b3cc79 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:29.604220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548f7c1-39ca-40b7-8646-0aae0483cb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:29.604822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548f7c1-39ca-40b7-8646-0aae0483cb55 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:29.605098 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548f7c1-39ca-40b7-8646-0aae0483cb55 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:29.605366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548f7c1-39ca-40b7-8646-0aae0483cb55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:29.606075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548f7c1-39ca-40b7-8646-0aae0483cb55 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.domain.id=default, user.id=f6a4ad480e8041b39ccfa602ac02988c) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:29.606374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548f7c1-39ca-40b7-8646-0aae0483cb55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:29.607855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548f7c1-39ca-40b7-8646-0aae0483cb55 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:29.614319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 11/20] 10.222.0.105 () {56 vars in 1327 bytes} [Wed Aug 24 04:40:29 2022] GET /identity/v3/role_assignments?user.id=f6a4ad480e8041b39ccfa602ac02988c&scope.domain.id=default&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 234 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:29.634584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f82363-6499-4646-b93a-f15139e80a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:29.635264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f82363-6499-4646-b93a-f15139e80a27 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:29.635571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f82363-6499-4646-b93a-f15139e80a27 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:29.635828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f82363-6499-4646-b93a-f15139e80a27 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:29.636500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f82363-6499-4646-b93a-f15139e80a27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:29.636807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f82363-6499-4646-b93a-f15139e80a27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:29.638270 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f82363-6499-4646-b93a-f15139e80a27 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:29.658246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 10/21] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:40:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:31.791552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9c248-98e2-4908-8fee-39d924c194f2 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:31.792055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9c248-98e2-4908-8fee-39d924c194f2 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:31.792127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9c248-98e2-4908-8fee-39d924c194f2 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:31.794143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 12/22] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:31.799635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cadb2-ec7f-4b24-b638-f2ba9ff24333 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:31.800056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cadb2-ec7f-4b24-b638-f2ba9ff24333 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:31.800240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cadb2-ec7f-4b24-b638-f2ba9ff24333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:31.820180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-145cadb2-ec7f-4b24-b638-f2ba9ff24333 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:31.824609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-145cadb2-ec7f-4b24-b638-f2ba9ff24333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:31.892701 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 11/23] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:31.969976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e7fcd-8f3d-4979-80a7-774c957dd1e7 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:31.970587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e7fcd-8f3d-4979-80a7-774c957dd1e7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:31.971085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e7fcd-8f3d-4979-80a7-774c957dd1e7 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:31.972915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 13/24] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:31 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:31.979052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5e683-d6bf-4d9f-a947-886c230a32e2 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:31.979571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5e683-d6bf-4d9f-a947-886c230a32e2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:31.980042 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc5e683-d6bf-4d9f-a947-886c230a32e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:32.004040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5bc5e683-d6bf-4d9f-a947-886c230a32e2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:32.005579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5bc5e683-d6bf-4d9f-a947-886c230a32e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:32.057013 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:32.089355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 12/25] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:40:32.358618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2c3f-8011-42e5-b6a2-dd0b55a5b989 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:32.359048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2c3f-8011-42e5-b6a2-dd0b55a5b989 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:32.359551 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb2c3f-8011-42e5-b6a2-dd0b55a5b989 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:32.361109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 14/26] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:32.366736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e06ad-e4c1-41eb-ad3d-48518f3a52d0 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:32.367272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e06ad-e4c1-41eb-ad3d-48518f3a52d0 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:32.367763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e06ad-e4c1-41eb-ad3d-48518f3a52d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:32.386611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-fb4e06ad-e4c1-41eb-ad3d-48518f3a52d0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:32.388217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fb4e06ad-e4c1-41eb-ad3d-48518f3a52d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:32.399913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb2c3f-8011-42e5-b6a2-dd0b55a5b989 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:32.536935 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb2c3f-8011-42e5-b6a2-dd0b55a5b989 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:32.542590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 13/27] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:32.543899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9276d1-42a2-432f-9a8a-b3f108f23b51 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:32.545647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9276d1-42a2-432f-9a8a-b3f108f23b51 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:32.545647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9276d1-42a2-432f-9a8a-b3f108f23b51 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:32.552020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 14/28] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:32.552020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9276d1-42a2-432f-9a8a-b3f108f23b51 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:32.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a18f25-79fd-4c95-8f48-0d57038b5bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:32.615974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a18f25-79fd-4c95-8f48-0d57038b5bf5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:32.616504 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a18f25-79fd-4c95-8f48-0d57038b5bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:32.616971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a18f25-79fd-4c95-8f48-0d57038b5bf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:32.634667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a18f25-79fd-4c95-8f48-0d57038b5bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:32.636487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a18f25-79fd-4c95-8f48-0d57038b5bf5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:32.645006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-07a18f25-79fd-4c95-8f48-0d57038b5bf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:40:32.659828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:32.672594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:40:32.673434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:40:32.673434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:40:32.673434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:32.673434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 15/29] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:40:32.673434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db353a83-3417-46f0-b06b-5db8d1273eb7 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:32.673434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db353a83-3417-46f0-b06b-5db8d1273eb7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:32.673434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db353a83-3417-46f0-b06b-5db8d1273eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:32.681125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9276d1-42a2-432f-9a8a-b3f108f23b51 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:32.694891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:32.704167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-db353a83-3417-46f0-b06b-5db8d1273eb7 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:32.707992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-db353a83-3417-46f0-b06b-5db8d1273eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:32.725159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a7d11-71a2-42ad-9722-08169ec571c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:32.726834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a7d11-71a2-42ad-9722-08169ec571c8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:32.727199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a7d11-71a2-42ad-9722-08169ec571c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:32.727477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89a7d11-71a2-42ad-9722-08169ec571c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:32.741117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89a7d11-71a2-42ad-9722-08169ec571c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-c89a7d11-71a2-42ad-9722-08169ec571c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:32.749187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:32.750059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 15/30] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:40:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:32.751757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a7d11-71a2-42ad-9722-08169ec571c8 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:32.867815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89a7d11-71a2-42ad-9722-08169ec571c8 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:32.880692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 16/31] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:32.881563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db353a83-3417-46f0-b06b-5db8d1273eb7 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:32.923396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22982181-6d53-41bf-a666-d75e5f152bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:32.924007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22982181-6d53-41bf-a666-d75e5f152bd5 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:32.924346 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22982181-6d53-41bf-a666-d75e5f152bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:32.924620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22982181-6d53-41bf-a666-d75e5f152bd5 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:32.925271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22982181-6d53-41bf-a666-d75e5f152bd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:32.925568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22982181-6d53-41bf-a666-d75e5f152bd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:32.926948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22982181-6d53-41bf-a666-d75e5f152bd5 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:32.940208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 16/32] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:40:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:32.941028 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22982181-6d53-41bf-a666-d75e5f152bd5 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:32.974904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:33.058010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db353a83-3417-46f0-b06b-5db8d1273eb7 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:33.077838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22982181-6d53-41bf-a666-d75e5f152bd5 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:33.086913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:33.097505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1042a-25e8-49ce-80b6-3ef6c3affa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:33.098441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1042a-25e8-49ce-80b6-3ef6c3affa46 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:33.098960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1042a-25e8-49ce-80b6-3ef6c3affa46 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:33.099504 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1042a-25e8-49ce-80b6-3ef6c3affa46 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:33.100383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1042a-25e8-49ce-80b6-3ef6c3affa46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:33.100880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1042a-25e8-49ce-80b6-3ef6c3affa46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:33.102475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1042a-25e8-49ce-80b6-3ef6c3affa46 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:33.116161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 17/33] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:32 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:33.117417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1042a-25e8-49ce-80b6-3ef6c3affa46 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:33.164811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c848e3-1af9-4a59-a371-83ecd74cf7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:33.166596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c848e3-1af9-4a59-a371-83ecd74cf7e1 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:33.168054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c848e3-1af9-4a59-a371-83ecd74cf7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:33.168722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c848e3-1af9-4a59-a371-83ecd74cf7e1 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:33.170763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c848e3-1af9-4a59-a371-83ecd74cf7e1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:33.172619 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c848e3-1af9-4a59-a371-83ecd74cf7e1 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:33.259690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1042a-25e8-49ce-80b6-3ef6c3affa46 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:33.301275 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b79a5-979b-423e-8dc2-b9b9faf3d186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:33.302262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b79a5-979b-423e-8dc2-b9b9faf3d186 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:33.302763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b79a5-979b-423e-8dc2-b9b9faf3d186 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:33.303264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b79a5-979b-423e-8dc2-b9b9faf3d186 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:33.304184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b79a5-979b-423e-8dc2-b9b9faf3d186 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:33.305935 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b79a5-979b-423e-8dc2-b9b9faf3d186 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:33.407049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 17/34] 10.222.0.105 () {60 vars in 1136 bytes} [Wed Aug 24 04:40:32 2022] POST /identity/v3/roles => generated 220 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:40:33.407049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c848e3-1af9-4a59-a371-83ecd74cf7e1 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:33.512183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:33.515410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 18/35] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:40:33 2022] POST /identity/v3/projects => generated 302 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:40:33.516688 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b79a5-979b-423e-8dc2-b9b9faf3d186 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:33.530753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c848e3-1af9-4a59-a371-83ecd74cf7e1 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:33.596314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6573a-daf0-4bbf-9cd1-fffa0e65c0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:33.597755 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6573a-daf0-4bbf-9cd1-fffa0e65c0ea None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:33.598408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6573a-daf0-4bbf-9cd1-fffa0e65c0ea None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:33.598981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6573a-daf0-4bbf-9cd1-fffa0e65c0ea None admin] PATH_INFO: `/v3/users/f6a4ad480e8041b39ccfa602ac02988c` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:33.601316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6573a-daf0-4bbf-9cd1-fffa0e65c0ea None admin] RBAC: Authorizing `identity:get_user(user_id=f6a4ad480e8041b39ccfa602ac02988c)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:33.603432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:33.604181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6573a-daf0-4bbf-9cd1-fffa0e65c0ea None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:33.610895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 18/36] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:33 2022] GET /identity/v3/users/f6a4ad480e8041b39ccfa602ac02988c => generated 248 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:33.612524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe2d3b-98b8-4962-b8e0-562610eeb2cd None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:33.612825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe2d3b-98b8-4962-b8e0-562610eeb2cd None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:33.613137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fe2d3b-98b8-4962-b8e0-562610eeb2cd None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:33.619056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 19/37] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:33 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:33.619056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa332e15-610e-4aba-a145-31ca5010ef69 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:33.619056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa332e15-610e-4aba-a145-31ca5010ef69 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:33.619056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa332e15-610e-4aba-a145-31ca5010ef69 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:33.623210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 20/38] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:33.623210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa332e15-610e-4aba-a145-31ca5010ef69 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:33.683174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b79a5-979b-423e-8dc2-b9b9faf3d186 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:33.728946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c825c44-544e-4968-a612-a7a41bb18a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:33.729577 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c825c44-544e-4968-a612-a7a41bb18a41 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:33.729856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c825c44-544e-4968-a612-a7a41bb18a41 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:33.730124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c825c44-544e-4968-a612-a7a41bb18a41 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:33.730791 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c825c44-544e-4968-a612-a7a41bb18a41 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:33.732345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c825c44-544e-4968-a612-a7a41bb18a41 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:33.752257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa332e15-610e-4aba-a145-31ca5010ef69 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:33.786178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140feffd-8826-41b8-ba59-ebbb3722ff18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:33.786749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140feffd-8826-41b8-ba59-ebbb3722ff18 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:33.787056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140feffd-8826-41b8-ba59-ebbb3722ff18 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:33.787330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140feffd-8826-41b8-ba59-ebbb3722ff18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:33.788533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140feffd-8826-41b8-ba59-ebbb3722ff18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:33.789978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140feffd-8826-41b8-ba59-ebbb3722ff18 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:33.791567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 21/39] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:33 2022] GET /identity/v3/domains/default => generated 200 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:33.792738 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262a43c-d23f-4b77-add6-4c987c8ac604 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:33.793018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262a43c-d23f-4b77-add6-4c987c8ac604 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:33.793292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262a43c-d23f-4b77-add6-4c987c8ac604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:33.808723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 19/40] 10.222.0.105 () {60 vars in 1136 bytes} [Wed Aug 24 04:40:33 2022] POST /identity/v3/roles => generated 226 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:40:33.810077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83657a50-9e79-40b2-a606-8b6fad302999 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:33.810367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83657a50-9e79-40b2-a606-8b6fad302999 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:33.810640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83657a50-9e79-40b2-a606-8b6fad302999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:33.811625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f262a43c-d23f-4b77-add6-4c987c8ac604 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:33.813302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f262a43c-d23f-4b77-add6-4c987c8ac604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:33.831614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-83657a50-9e79-40b2-a606-8b6fad302999 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:33.833708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-83657a50-9e79-40b2-a606-8b6fad302999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:33.834095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:33.892433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 22/41] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:33.903753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262a43c-d23f-4b77-add6-4c987c8ac604 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:33.918794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 20/42] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:34.015388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262a43c-d23f-4b77-add6-4c987c8ac604 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:34.051897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1d5dd-328c-481c-8759-ce4f8d98cc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:34.052562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1d5dd-328c-481c-8759-ce4f8d98cc3b None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:34.052842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1d5dd-328c-481c-8759-ce4f8d98cc3b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:34.053117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1d5dd-328c-481c-8759-ce4f8d98cc3b None admin] PATH_INFO: `/v3/domains/default/users/f6a4ad480e8041b39ccfa602ac02988c/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:34.055179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1d5dd-328c-481c-8759-ce4f8d98cc3b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f6a4ad480e8041b39ccfa602ac02988c, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:34.056623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1d5dd-328c-481c-8759-ce4f8d98cc3b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:34.166239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83657a50-9e79-40b2-a606-8b6fad302999 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:34.169369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 23/43] 10.222.0.105 () {58 vars in 1366 bytes} [Wed Aug 24 04:40:33 2022] PUT /identity/v3/domains/default/users/f6a4ad480e8041b39ccfa602ac02988c/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:40:34.188346 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1d5dd-328c-481c-8759-ce4f8d98cc3b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:34.207776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:34.209478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:34.287527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83657a50-9e79-40b2-a606-8b6fad302999 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:34.306459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1d5dd-328c-481c-8759-ce4f8d98cc3b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:34.333668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ebadd1-9feb-4e0a-9da2-11da46f76ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:34.334295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebadd1-9feb-4e0a-9da2-11da46f76ab1 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:34.334575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebadd1-9feb-4e0a-9da2-11da46f76ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:34.334845 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ebadd1-9feb-4e0a-9da2-11da46f76ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:34.335548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebadd1-9feb-4e0a-9da2-11da46f76ab1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:34.336975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ebadd1-9feb-4e0a-9da2-11da46f76ab1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:34.349917 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0008f8-8ae1-459b-95cc-daf9b1751b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:34.351216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0008f8-8ae1-459b-95cc-daf9b1751b3b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:34.351216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0008f8-8ae1-459b-95cc-daf9b1751b3b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:34.351216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0008f8-8ae1-459b-95cc-daf9b1751b3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:34.352202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0008f8-8ae1-459b-95cc-daf9b1751b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:34.353646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0008f8-8ae1-459b-95cc-daf9b1751b3b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:34.355244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 24/44] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:34 2022] GET /identity/v3/domains/default => generated 200 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:34.360736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0008f8-8ae1-459b-95cc-daf9b1751b3b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:34.404496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 21/45] 10.222.0.105 () {60 vars in 1136 bytes} [Wed Aug 24 04:40:34 2022] POST /identity/v3/roles => generated 224 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:40:34.555568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:34.575856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0008f8-8ae1-459b-95cc-daf9b1751b3b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:34.614715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698bedb5-4744-478c-bbac-2faed7788e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:34.615430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bedb5-4744-478c-bbac-2faed7788e7a None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:34.615714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bedb5-4744-478c-bbac-2faed7788e7a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:34.615989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698bedb5-4744-478c-bbac-2faed7788e7a None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:34.616644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bedb5-4744-478c-bbac-2faed7788e7a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:34.618093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698bedb5-4744-478c-bbac-2faed7788e7a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:34.696470 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 25/46] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:40:34 2022] POST /identity/v3/projects => generated 313 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:40:36.105412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf748a53-096c-4b8b-8a84-5e7d96e73828 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:36.105935 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf748a53-096c-4b8b-8a84-5e7d96e73828 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:36.106025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf748a53-096c-4b8b-8a84-5e7d96e73828 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:36.107365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 22/47] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:36.113288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed7c99-2603-4f1f-9516-ed583d1debb2 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:36.113719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed7c99-2603-4f1f-9516-ed583d1debb2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:36.113847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed7c99-2603-4f1f-9516-ed583d1debb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:36.132071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-27ed7c99-2603-4f1f-9516-ed583d1debb2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:36.133305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-27ed7c99-2603-4f1f-9516-ed583d1debb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:36.201309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 26/48] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:40:36.474395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf748a53-096c-4b8b-8a84-5e7d96e73828 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:36.567438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:36.577654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf748a53-096c-4b8b-8a84-5e7d96e73828 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:36.630506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89c46-eeae-4e5b-a54f-04fb473adac1 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:36.630801 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89c46-eeae-4e5b-a54f-04fb473adac1 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:36.631113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e89c46-eeae-4e5b-a54f-04fb473adac1 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:36.632500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 27/49] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:40:36.638564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba53fed-1a9f-4743-a4f8-5656b44811b6 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:36.638846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba53fed-1a9f-4743-a4f8-5656b44811b6 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:36.639162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba53fed-1a9f-4743-a4f8-5656b44811b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:36.659554 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5ba53fed-1a9f-4743-a4f8-5656b44811b6 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:36.660786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5ba53fed-1a9f-4743-a4f8-5656b44811b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:36.717830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd95c1-7f73-487c-a4c3-ca844ce035f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:36.718459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd95c1-7f73-487c-a4c3-ca844ce035f9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:36.718734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd95c1-7f73-487c-a4c3-ca844ce035f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:36.719004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd95c1-7f73-487c-a4c3-ca844ce035f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:36.720425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:36.734184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd95c1-7f73-487c-a4c3-ca844ce035f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:36.734317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 28/50] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-8cdd95c1-7f73-487c-a4c3-ca844ce035f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:36.735366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:36.736295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:36.737782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 23/51] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:40:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:36.742770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba53fed-1a9f-4743-a4f8-5656b44811b6 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:36.839962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba53fed-1a9f-4743-a4f8-5656b44811b6 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:36.874015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547511ec-64f5-4edc-b90c-5fb4b164a6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:36.874614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547511ec-64f5-4edc-b90c-5fb4b164a6e0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:36.874892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547511ec-64f5-4edc-b90c-5fb4b164a6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:36.875198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547511ec-64f5-4edc-b90c-5fb4b164a6e0 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:36.875857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547511ec-64f5-4edc-b90c-5fb4b164a6e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:36.876155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547511ec-64f5-4edc-b90c-5fb4b164a6e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:36.878091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547511ec-64f5-4edc-b90c-5fb4b164a6e0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:36.889509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 29/52] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:36 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:36.896136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd95c1-7f73-487c-a4c3-ca844ce035f9 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:36.994732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd95c1-7f73-487c-a4c3-ca844ce035f9 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:37.004725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547511ec-64f5-4edc-b90c-5fb4b164a6e0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:37.029341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2c14a-5664-4a3e-b27e-ef190e92fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:37.029970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2c14a-5664-4a3e-b27e-ef190e92fad9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:37.030251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2c14a-5664-4a3e-b27e-ef190e92fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:37.030518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2c14a-5664-4a3e-b27e-ef190e92fad9 None admin] PATH_INFO: `/v3/users/f6a4ad480e8041b39ccfa602ac02988c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:37.031657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2c14a-5664-4a3e-b27e-ef190e92fad9 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a4ad480e8041b39ccfa602ac02988c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:37.033238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2c14a-5664-4a3e-b27e-ef190e92fad9 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:37.035906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 24/53] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:36 2022] GET /identity/v3/users/f6a4ad480e8041b39ccfa602ac02988c => generated 248 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:37.041380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2c14a-5664-4a3e-b27e-ef190e92fad9 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:37.060582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:37.061791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:37.106798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547511ec-64f5-4edc-b90c-5fb4b164a6e0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:37.140783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2c14a-5664-4a3e-b27e-ef190e92fad9 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:37.141313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb104e1-a726-4289-83ea-858cdd8d6970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:37.141683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb104e1-a726-4289-83ea-858cdd8d6970 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:37.141941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb104e1-a726-4289-83ea-858cdd8d6970 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:37.142215 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb104e1-a726-4289-83ea-858cdd8d6970 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:37.143889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb104e1-a726-4289-83ea-858cdd8d6970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:37.145909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb104e1-a726-4289-83ea-858cdd8d6970 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:37.147517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 30/54] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:37 2022] GET /identity/v3/domains/default => generated 200 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:37.153181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb104e1-a726-4289-83ea-858cdd8d6970 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:37.176329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779bd10-1ea2-4977-9a54-4836658ccddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:37.176972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779bd10-1ea2-4977-9a54-4836658ccddb None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:37.177257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779bd10-1ea2-4977-9a54-4836658ccddb None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:37.177534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779bd10-1ea2-4977-9a54-4836658ccddb None admin] PATH_INFO: `/v3/domains/default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:37.178674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779bd10-1ea2-4977-9a54-4836658ccddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:37.180204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e779bd10-1ea2-4977-9a54-4836658ccddb None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:37.181770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 25/55] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:37 2022] GET /identity/v3/domains/default => generated 200 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:37.187486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779bd10-1ea2-4977-9a54-4836658ccddb None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:37.257032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb104e1-a726-4289-83ea-858cdd8d6970 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:37.272860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:37.280535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:37.288255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779bd10-1ea2-4977-9a54-4836658ccddb None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:37.291905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07c102-3cd4-4676-b548-47d001ed9d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:37.292514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07c102-3cd4-4676-b548-47d001ed9d85 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:37.292801 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07c102-3cd4-4676-b548-47d001ed9d85 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:37.293078 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07c102-3cd4-4676-b548-47d001ed9d85 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:37.293779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07c102-3cd4-4676-b548-47d001ed9d85 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:37.295302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07c102-3cd4-4676-b548-47d001ed9d85 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:37.322218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fefee0-1b1b-49b7-a55a-c32570014694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:37.322828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fefee0-1b1b-49b7-a55a-c32570014694 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:37.323163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fefee0-1b1b-49b7-a55a-c32570014694 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:37.323444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fefee0-1b1b-49b7-a55a-c32570014694 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:37.324165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fefee0-1b1b-49b7-a55a-c32570014694 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.domain.id=default, user.id=f6a4ad480e8041b39ccfa602ac02988c) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:37.324476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fefee0-1b1b-49b7-a55a-c32570014694 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:37.325914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fefee0-1b1b-49b7-a55a-c32570014694 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:37.332180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 26/56] 10.222.0.105 () {56 vars in 1327 bytes} [Wed Aug 24 04:40:37 2022] GET /identity/v3/role_assignments?user.id=f6a4ad480e8041b39ccfa602ac02988c&scope.domain.id=default&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 532 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:37.395422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 31/57] 10.222.0.105 () {60 vars in 1145 bytes} [Wed Aug 24 04:40:37 2022] POST /identity/v3/projects => generated 299 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:40:39.307288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c3658-c837-4c56-8013-8f6a2bb59862 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:39.307802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c3658-c837-4c56-8013-8f6a2bb59862 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:39.307894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0c3658-c837-4c56-8013-8f6a2bb59862 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:39.309180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 27/58] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:39.314463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58ec55-8685-4187-b920-cf224196e702 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:39.314830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58ec55-8685-4187-b920-cf224196e702 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:39.315038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58ec55-8685-4187-b920-cf224196e702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:39.332585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5a58ec55-8685-4187-b920-cf224196e702 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:39.334444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5a58ec55-8685-4187-b920-cf224196e702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:39.400313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 32/59] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:39.679162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c3658-c837-4c56-8013-8f6a2bb59862 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:39.781234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0c3658-c837-4c56-8013-8f6a2bb59862 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:39.815264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5bf54-a1e9-4631-9bed-2eec2481696e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:39.815882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5bf54-a1e9-4631-9bed-2eec2481696e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:39.816164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5bf54-a1e9-4631-9bed-2eec2481696e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:39.816440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5bf54-a1e9-4631-9bed-2eec2481696e None admin] PATH_INFO: `/v3/domains/default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:39.817557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5bf54-a1e9-4631-9bed-2eec2481696e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:39.819335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5bf54-a1e9-4631-9bed-2eec2481696e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:39.820888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 28/60] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:39 2022] GET /identity/v3/domains/default => generated 200 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:39.826464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58ec55-8685-4187-b920-cf224196e702 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:39.923109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58ec55-8685-4187-b920-cf224196e702 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:39.925331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:39.955602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276928c-1d9b-4dd6-9325-10a287ef03bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:39.956146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276928c-1d9b-4dd6-9325-10a287ef03bb None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:39.956435 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276928c-1d9b-4dd6-9325-10a287ef03bb None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:39.956715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276928c-1d9b-4dd6-9325-10a287ef03bb None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:39.957360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276928c-1d9b-4dd6-9325-10a287ef03bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:39.958815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276928c-1d9b-4dd6-9325-10a287ef03bb None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:40.098840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 33/61] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:40:39 2022] POST /identity/v3/users => generated 276 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:40:42.940698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ecc94-2202-46bf-9cff-673c659983f1 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:42.941739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ecc94-2202-46bf-9cff-673c659983f1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:42.942234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ecc94-2202-46bf-9cff-673c659983f1 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:42.943896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 29/62] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:42 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:42.950117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78cb08-887a-4a09-8b6c-1a689a7f5809 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:42.950979 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78cb08-887a-4a09-8b6c-1a689a7f5809 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:42.951502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78cb08-887a-4a09-8b6c-1a689a7f5809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:42.974733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2b78cb08-887a-4a09-8b6c-1a689a7f5809 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:42.976292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2b78cb08-887a-4a09-8b6c-1a689a7f5809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:43.076717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 34/63] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:43.398735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed9505-9620-4438-b7ca-09af971e27ce None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:43.399442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed9505-9620-4438-b7ca-09af971e27ce None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:43.400575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ed9505-9620-4438-b7ca-09af971e27ce None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:43.402114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 30/64] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:43.407975 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c533175-9fd3-4154-bf44-40554a375786 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:43.408508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c533175-9fd3-4154-bf44-40554a375786 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:43.408982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c533175-9fd3-4154-bf44-40554a375786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:43.433286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed9505-9620-4438-b7ca-09af971e27ce None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:43.453670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-0c533175-9fd3-4154-bf44-40554a375786 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:43.456464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0c533175-9fd3-4154-bf44-40554a375786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:43.462374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:43.585131 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ed9505-9620-4438-b7ca-09af971e27ce None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:43.585663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 35/65] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:43.587439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d842f-c1d5-4bd3-bafd-f2b0df7a81ff None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:43.588777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d842f-c1d5-4bd3-bafd-f2b0df7a81ff None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:43.589437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d842f-c1d5-4bd3-bafd-f2b0df7a81ff None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:43.590956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 36/66] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:43 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:43.596280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704430f9-a8fa-4df8-90e5-c54698f201d0 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:43.596905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704430f9-a8fa-4df8-90e5-c54698f201d0 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:43.597375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704430f9-a8fa-4df8-90e5-c54698f201d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:43.617015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-704430f9-a8fa-4df8-90e5-c54698f201d0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:43.618561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-704430f9-a8fa-4df8-90e5-c54698f201d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:43.626356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ca3102-6762-4aba-85c8-93353d75f968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:43.626989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca3102-6762-4aba-85c8-93353d75f968 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:43.627294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca3102-6762-4aba-85c8-93353d75f968 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:43.627563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ca3102-6762-4aba-85c8-93353d75f968 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:43.638373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:43.641930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ca3102-6762-4aba-85c8-93353d75f968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-62ca3102-6762-4aba-85c8-93353d75f968 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:43.647772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:43.653982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 31/67] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:43.654759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f32e0-5cd2-41f8-ab5a-eb5cb07f1938 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:43.655910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f32e0-5cd2-41f8-ab5a-eb5cb07f1938 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:43.655910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f32e0-5cd2-41f8-ab5a-eb5cb07f1938 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:43.655910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 32/68] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:43.659732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a479894-6c4e-46eb-99dc-2a27c0bfed2d None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:43.660176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a479894-6c4e-46eb-99dc-2a27c0bfed2d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:43.660585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a479894-6c4e-46eb-99dc-2a27c0bfed2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:43.679693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9a479894-6c4e-46eb-99dc-2a27c0bfed2d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:43.681085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9a479894-6c4e-46eb-99dc-2a27c0bfed2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:43.703685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 37/69] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:40:43.704688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704430f9-a8fa-4df8-90e5-c54698f201d0 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:43.766824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 33/70] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:43.769204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6863-f1bc-454e-b443-8471eb54bb8e None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:43.769687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6863-f1bc-454e-b443-8471eb54bb8e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:43.770519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22db6863-f1bc-454e-b443-8471eb54bb8e None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:43.772375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 34/71] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:43 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:43.784231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550c2d-1e45-402e-8fee-d07c95a7bbb1 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:43.784764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550c2d-1e45-402e-8fee-d07c95a7bbb1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:43.785230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550c2d-1e45-402e-8fee-d07c95a7bbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:43.816135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704430f9-a8fa-4df8-90e5-c54698f201d0 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:43.827812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-51550c2d-1e45-402e-8fee-d07c95a7bbb1 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:43.829134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-51550c2d-1e45-402e-8fee-d07c95a7bbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:43.831302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:43.854673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79b1e-1f91-409d-9119-0fc051e1019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:43.855671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79b1e-1f91-409d-9119-0fc051e1019b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:43.856150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79b1e-1f91-409d-9119-0fc051e1019b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:43.856623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc79b1e-1f91-409d-9119-0fc051e1019b None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:43.857431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc79b1e-1f91-409d-9119-0fc051e1019b None admin] RBAC: Adding query filter params (name=member) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:43.857912 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc79b1e-1f91-409d-9119-0fc051e1019b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:43.859507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc79b1e-1f91-409d-9119-0fc051e1019b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:43.868777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 38/72] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:40:43 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:43.875235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79b1e-1f91-409d-9119-0fc051e1019b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:43.891889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:43.976170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 35/73] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:43 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:43.977025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51550c2d-1e45-402e-8fee-d07c95a7bbb1 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:43.981499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc79b1e-1f91-409d-9119-0fc051e1019b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.018085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3343b-1243-45ac-b746-e0a1c2259553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.019127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3343b-1243-45ac-b746-e0a1c2259553 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.019605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3343b-1243-45ac-b746-e0a1c2259553 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.020099 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e3343b-1243-45ac-b746-e0a1c2259553 None admin] PATH_INFO: `/v3/users/5109071b2f5d4a8fbd1639a297f22e81` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.036443 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3343b-1243-45ac-b746-e0a1c2259553 None admin] RBAC: Authorizing `identity:get_user(user_id=5109071b2f5d4a8fbd1639a297f22e81)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:44.038278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e3343b-1243-45ac-b746-e0a1c2259553 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:44.040323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 39/74] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:43 2022] GET /identity/v3/users/5109071b2f5d4a8fbd1639a297f22e81 => generated 276 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:44.041810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3343b-1243-45ac-b746-e0a1c2259553 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.113288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51550c2d-1e45-402e-8fee-d07c95a7bbb1 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.114862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:44.147238 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e3343b-1243-45ac-b746-e0a1c2259553 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.150199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eea9a-166e-40ac-afaa-c702a4270957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.151122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eea9a-166e-40ac-afaa-c702a4270957 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.151424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eea9a-166e-40ac-afaa-c702a4270957 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.151774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478eea9a-166e-40ac-afaa-c702a4270957 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.159600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:44.164051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478eea9a-166e-40ac-afaa-c702a4270957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-478eea9a-166e-40ac-afaa-c702a4270957 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:44.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:44.170565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:44.171696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 36/75] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:44.172333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eea9a-166e-40ac-afaa-c702a4270957 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.192628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b876d93-59bd-409b-b640-f54d1ec15d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.193285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b876d93-59bd-409b-b640-f54d1ec15d36 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.193547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b876d93-59bd-409b-b640-f54d1ec15d36 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.193818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b876d93-59bd-409b-b640-f54d1ec15d36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.205190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b876d93-59bd-409b-b640-f54d1ec15d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-1b876d93-59bd-409b-b640-f54d1ec15d36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:44.210640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:44.211549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:44.212325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 40/76] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:44.213423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b876d93-59bd-409b-b640-f54d1ec15d36 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.279428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478eea9a-166e-40ac-afaa-c702a4270957 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.313624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88e77-fc0c-4b93-b887-8670b2fb91b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.314257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88e77-fc0c-4b93-b887-8670b2fb91b5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.314537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88e77-fc0c-4b93-b887-8670b2fb91b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.314814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d88e77-fc0c-4b93-b887-8670b2fb91b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.317433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b876d93-59bd-409b-b640-f54d1ec15d36 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.324937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d88e77-fc0c-4b93-b887-8670b2fb91b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-47d88e77-fc0c-4b93-b887-8670b2fb91b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:44.329677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:44.330706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 37/77] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:44.331658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88e77-fc0c-4b93-b887-8670b2fb91b5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.351027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:44.357289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a22891-eeef-43dd-85f2-894fb407f1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.357940 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a22891-eeef-43dd-85f2-894fb407f1dd None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.358220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a22891-eeef-43dd-85f2-894fb407f1dd None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.358489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a22891-eeef-43dd-85f2-894fb407f1dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.389101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:44.426583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a22891-eeef-43dd-85f2-894fb407f1dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:44.428086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a22891-eeef-43dd-85f2-894fb407f1dd None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:44.430026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d88e77-fc0c-4b93-b887-8670b2fb91b5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.461859 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75bd87-8721-43af-9b6e-885355fc69ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.462459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75bd87-8721-43af-9b6e-885355fc69ca None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.462743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75bd87-8721-43af-9b6e-885355fc69ca None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.463037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75bd87-8721-43af-9b6e-885355fc69ca None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.464193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75bd87-8721-43af-9b6e-885355fc69ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:44.464498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75bd87-8721-43af-9b6e-885355fc69ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:44.465886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75bd87-8721-43af-9b6e-885355fc69ca None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:44.474253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 38/78] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:44.479000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75bd87-8721-43af-9b6e-885355fc69ca None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.530916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 41/79] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:40:44.531544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a22891-eeef-43dd-85f2-894fb407f1dd None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.560806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:44.578332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75bd87-8721-43af-9b6e-885355fc69ca None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.597946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:44.612293 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f232d96-0b96-4bbd-b0c6-2cfb25188d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.612898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f232d96-0b96-4bbd-b0c6-2cfb25188d9b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.613177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f232d96-0b96-4bbd-b0c6-2cfb25188d9b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.613445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f232d96-0b96-4bbd-b0c6-2cfb25188d9b None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.614101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f232d96-0b96-4bbd-b0c6-2cfb25188d9b None admin] RBAC: Adding query filter params (name=member) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:44.614649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f232d96-0b96-4bbd-b0c6-2cfb25188d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:44.615838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f232d96-0b96-4bbd-b0c6-2cfb25188d9b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:44.624504 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 39/80] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:44.625369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f232d96-0b96-4bbd-b0c6-2cfb25188d9b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.631529 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a22891-eeef-43dd-85f2-894fb407f1dd None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.665151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ad20e-e360-41f4-8370-6c651ad056f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.665778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ad20e-e360-41f4-8370-6c651ad056f0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.666062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ad20e-e360-41f4-8370-6c651ad056f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.666342 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ad20e-e360-41f4-8370-6c651ad056f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.668174 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ad20e-e360-41f4-8370-6c651ad056f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:44.669544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ad20e-e360-41f4-8370-6c651ad056f0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:44.671125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 42/81] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/domains/default => generated 200 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:44.671890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ad20e-e360-41f4-8370-6c651ad056f0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.724271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f232d96-0b96-4bbd-b0c6-2cfb25188d9b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.757741 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c04f1-2d20-4e7e-a678-b675463a8c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.758350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c04f1-2d20-4e7e-a678-b675463a8c69 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.758632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c04f1-2d20-4e7e-a678-b675463a8c69 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.758906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c04f1-2d20-4e7e-a678-b675463a8c69 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.759605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c04f1-2d20-4e7e-a678-b675463a8c69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:44.759912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c04f1-2d20-4e7e-a678-b675463a8c69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:44.762240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c04f1-2d20-4e7e-a678-b675463a8c69 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:44.766537 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:44.770055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 40/82] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:44.770850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c04f1-2d20-4e7e-a678-b675463a8c69 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.771748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ad20e-e360-41f4-8370-6c651ad056f0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.806747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e97bdc-e6de-4c02-8d2b-141351eb2659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.807414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97bdc-e6de-4c02-8d2b-141351eb2659 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.808269 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97bdc-e6de-4c02-8d2b-141351eb2659 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.808546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e97bdc-e6de-4c02-8d2b-141351eb2659 None admin] PATH_INFO: `/v3/users/f6a4ad480e8041b39ccfa602ac02988c` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.809690 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e97bdc-e6de-4c02-8d2b-141351eb2659 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a4ad480e8041b39ccfa602ac02988c)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:44.811174 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e97bdc-e6de-4c02-8d2b-141351eb2659 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:44.812761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 43/83] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/users/f6a4ad480e8041b39ccfa602ac02988c => generated 248 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:44.813748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e97bdc-e6de-4c02-8d2b-141351eb2659 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.816527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:44.871289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c04f1-2d20-4e7e-a678-b675463a8c69 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.904947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaa2cb-e030-44e0-876b-5c9e29ed90a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.905576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa2cb-e030-44e0-876b-5c9e29ed90a5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.905885 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa2cb-e030-44e0-876b-5c9e29ed90a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.906159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aaa2cb-e030-44e0-876b-5c9e29ed90a5 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.907360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaa2cb-e030-44e0-876b-5c9e29ed90a5 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:44.909072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aaa2cb-e030-44e0-876b-5c9e29ed90a5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:44.910727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 41/84] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:44.915170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaa2cb-e030-44e0-876b-5c9e29ed90a5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.918127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e97bdc-e6de-4c02-8d2b-141351eb2659 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:44.951378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663431f4-7c14-4fba-b058-ea40e2ef6c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:44.952000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663431f4-7c14-4fba-b058-ea40e2ef6c9c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:44.952279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663431f4-7c14-4fba-b058-ea40e2ef6c9c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:44.952552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663431f4-7c14-4fba-b058-ea40e2ef6c9c None admin] PATH_INFO: `/v3/users/5109071b2f5d4a8fbd1639a297f22e81` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:44.953682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663431f4-7c14-4fba-b058-ea40e2ef6c9c None admin] RBAC: Authorizing `identity:get_user(user_id=5109071b2f5d4a8fbd1639a297f22e81)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:44.955159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663431f4-7c14-4fba-b058-ea40e2ef6c9c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:44.956717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 44/85] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/users/5109071b2f5d4a8fbd1639a297f22e81 => generated 276 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:44.959875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663431f4-7c14-4fba-b058-ea40e2ef6c9c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:44.994297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:45.009236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aaa2cb-e030-44e0-876b-5c9e29ed90a5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:45.022447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:45.045840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c9f2-9067-44f6-80c5-13de920128f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:45.046452 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c9f2-9067-44f6-80c5-13de920128f4 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:45.046734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c9f2-9067-44f6-80c5-13de920128f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:45.047014 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c9f2-9067-44f6-80c5-13de920128f4 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:45.047685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5c9f2-9067-44f6-80c5-13de920128f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:45.049138 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5c9f2-9067-44f6-80c5-13de920128f4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:45.060447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663431f4-7c14-4fba-b058-ea40e2ef6c9c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:45.095928 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e99668-047f-4261-a2fc-f8036f30f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:45.096546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e99668-047f-4261-a2fc-f8036f30f0ae None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:45.096824 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e99668-047f-4261-a2fc-f8036f30f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:45.097123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e99668-047f-4261-a2fc-f8036f30f0ae None admin] PATH_INFO: `/v3/users/5109071b2f5d4a8fbd1639a297f22e81` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:45.098259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e99668-047f-4261-a2fc-f8036f30f0ae None admin] RBAC: Authorizing `identity:get_user(user_id=5109071b2f5d4a8fbd1639a297f22e81)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:45.099715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e99668-047f-4261-a2fc-f8036f30f0ae None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:45.101294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 45/86] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:44 2022] GET /identity/v3/users/5109071b2f5d4a8fbd1639a297f22e81 => generated 276 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:45.102056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e99668-047f-4261-a2fc-f8036f30f0ae None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:45.134952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 42/87] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:40:44 2022] POST /identity/v3/users => generated 290 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:40:45.140048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c9f2-9067-44f6-80c5-13de920128f4 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:45.218654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e99668-047f-4261-a2fc-f8036f30f0ae None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:45.221174 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:45.258802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656af75-da4e-4439-bb8d-abb9e71be84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:45.259741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656af75-da4e-4439-bb8d-abb9e71be84c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:45.260027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656af75-da4e-4439-bb8d-abb9e71be84c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:45.260302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656af75-da4e-4439-bb8d-abb9e71be84c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:45.261804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c9f2-9067-44f6-80c5-13de920128f4 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:45.303584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646c6a0-97bd-4c45-92a8-6e36421c5fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:45.304200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646c6a0-97bd-4c45-92a8-6e36421c5fd1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:45.304481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646c6a0-97bd-4c45-92a8-6e36421c5fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:45.305406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646c6a0-97bd-4c45-92a8-6e36421c5fd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:45.305507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646c6a0-97bd-4c45-92a8-6e36421c5fd1 None admin] RBAC: Adding query filter params (role.id=df27fa65390e4de78fa117773dc73555, scope.project.id=a1f41b226cc94cccbb39608d62cb6737, user.id=5109071b2f5d4a8fbd1639a297f22e81) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:45.305769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646c6a0-97bd-4c45-92a8-6e36421c5fd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:45.307243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646c6a0-97bd-4c45-92a8-6e36421c5fd1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:45.313726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 43/88] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:40:45 2022] GET /identity/v3/role_assignments?user.id=5109071b2f5d4a8fbd1639a297f22e81&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=df27fa65390e4de78fa117773dc73555 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:45.314557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646c6a0-97bd-4c45-92a8-6e36421c5fd1 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:45.352139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656af75-da4e-4439-bb8d-abb9e71be84c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:45.353622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656af75-da4e-4439-bb8d-abb9e71be84c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:45.359443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:45.467599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646c6a0-97bd-4c45-92a8-6e36421c5fd1 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:45.494159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 46/89] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:45 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:40:45.495124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656af75-da4e-4439-bb8d-abb9e71be84c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:45.499977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:45.518526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd737023-8d5f-4167-892d-ec4edb544017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:45.519182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd737023-8d5f-4167-892d-ec4edb544017 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:45.519464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd737023-8d5f-4167-892d-ec4edb544017 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:45.519735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd737023-8d5f-4167-892d-ec4edb544017 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:45.595001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd737023-8d5f-4167-892d-ec4edb544017 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:45.596510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd737023-8d5f-4167-892d-ec4edb544017 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:45.616266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656af75-da4e-4439-bb8d-abb9e71be84c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:45.642966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:45.680865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db247968-cf52-4b7b-8d08-01b965ee4c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:45.681481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db247968-cf52-4b7b-8d08-01b965ee4c36 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:45.681758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db247968-cf52-4b7b-8d08-01b965ee4c36 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:45.682030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db247968-cf52-4b7b-8d08-01b965ee4c36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:45.741755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 44/90] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:45 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:40:45.743105 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd737023-8d5f-4167-892d-ec4edb544017 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:45.798671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db247968-cf52-4b7b-8d08-01b965ee4c36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:45.800056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db247968-cf52-4b7b-8d08-01b965ee4c36 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:45.802072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:45.874977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd737023-8d5f-4167-892d-ec4edb544017 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:45.908966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:45.914409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af6bc8-76e1-4eca-b9e1-12d2d494e2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:45.915300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af6bc8-76e1-4eca-b9e1-12d2d494e2ed None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:45.915767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af6bc8-76e1-4eca-b9e1-12d2d494e2ed None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:45.916198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af6bc8-76e1-4eca-b9e1-12d2d494e2ed None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:45.917660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af6bc8-76e1-4eca-b9e1-12d2d494e2ed None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:45.919531 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af6bc8-76e1-4eca-b9e1-12d2d494e2ed None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:45.921654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 45/91] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:45 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:45.922729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af6bc8-76e1-4eca-b9e1-12d2d494e2ed None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:45.962523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 47/92] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:45 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:40:45.963273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db247968-cf52-4b7b-8d08-01b965ee4c36 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:46.030151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af6bc8-76e1-4eca-b9e1-12d2d494e2ed None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:46.078020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecf0a8-065c-4605-8a37-d5b1b6512ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:46.079309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecf0a8-065c-4605-8a37-d5b1b6512ad8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:46.080136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecf0a8-065c-4605-8a37-d5b1b6512ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:46.080847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaecf0a8-065c-4605-8a37-d5b1b6512ad8 None admin] PATH_INFO: `/v3/projects/7cefd8a371b74c568801146b5a72469b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:46.083861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecf0a8-065c-4605-8a37-d5b1b6512ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=7cefd8a371b74c568801146b5a72469b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:46.085452 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaecf0a8-065c-4605-8a37-d5b1b6512ad8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:46.087591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 46/93] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:45 2022] GET /identity/v3/projects/7cefd8a371b74c568801146b5a72469b => generated 313 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:46.088553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecf0a8-065c-4605-8a37-d5b1b6512ad8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:46.150166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db247968-cf52-4b7b-8d08-01b965ee4c36 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:46.156324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:46.213122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:46.222651 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab5f9f-cb98-4768-b935-915110aa8f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:46.227314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab5f9f-cb98-4768-b935-915110aa8f52 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:46.227314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab5f9f-cb98-4768-b935-915110aa8f52 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:46.227314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab5f9f-cb98-4768-b935-915110aa8f52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:46.227314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab5f9f-cb98-4768-b935-915110aa8f52 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=a1f41b226cc94cccbb39608d62cb6737, user.id=f6a4ad480e8041b39ccfa602ac02988c) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:46.227314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab5f9f-cb98-4768-b935-915110aa8f52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:46.227314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab5f9f-cb98-4768-b935-915110aa8f52 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:46.245200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 48/94] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:40:45 2022] GET /identity/v3/role_assignments?user.id=f6a4ad480e8041b39ccfa602ac02988c&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 260 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:46.247542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab5f9f-cb98-4768-b935-915110aa8f52 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:46.280575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaecf0a8-065c-4605-8a37-d5b1b6512ad8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:46.336314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7da1ff-fce8-4ec5-9163-7940402233b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:46.336979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7da1ff-fce8-4ec5-9163-7940402233b3 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:46.337262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7da1ff-fce8-4ec5-9163-7940402233b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:46.337538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7da1ff-fce8-4ec5-9163-7940402233b3 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:46.339580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7da1ff-fce8-4ec5-9163-7940402233b3 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:46.341468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7da1ff-fce8-4ec5-9163-7940402233b3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:46.343368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 47/95] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:46 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:46.350964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7da1ff-fce8-4ec5-9163-7940402233b3 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:46.505938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab5f9f-cb98-4768-b935-915110aa8f52 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:46.547738 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1f4f0-702d-40fe-945a-88bf2999c36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:46.548702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f4f0-702d-40fe-945a-88bf2999c36e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:46.549172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f4f0-702d-40fe-945a-88bf2999c36e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:46.549712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f4f0-702d-40fe-945a-88bf2999c36e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:46.550712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1f4f0-702d-40fe-945a-88bf2999c36e None admin] RBAC: Adding query filter params (role.id=df27fa65390e4de78fa117773dc73555, scope.project.id=7cefd8a371b74c568801146b5a72469b, user.id=5109071b2f5d4a8fbd1639a297f22e81) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:46.551283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1f4f0-702d-40fe-945a-88bf2999c36e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:46.553545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1f4f0-702d-40fe-945a-88bf2999c36e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:46.557171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7da1ff-fce8-4ec5-9163-7940402233b3 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:46.560376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 49/96] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:40:46 2022] GET /identity/v3/role_assignments?user.id=5109071b2f5d4a8fbd1639a297f22e81&scope.project.id=7cefd8a371b74c568801146b5a72469b&role.id=df27fa65390e4de78fa117773dc73555 => generated 260 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:46.628922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1900f06-5983-4924-94e2-5bb71856d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:46.629567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1900f06-5983-4924-94e2-5bb71856d733 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:46.629849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1900f06-5983-4924-94e2-5bb71856d733 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:46.630120 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1900f06-5983-4924-94e2-5bb71856d733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:46.630871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1900f06-5983-4924-94e2-5bb71856d733 None admin] RBAC: Adding query filter params (role.id=b7f17471d2c948efa07d02424da68bea, scope.project.id=a1f41b226cc94cccbb39608d62cb6737, user.id=5109071b2f5d4a8fbd1639a297f22e81) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:46.631207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1900f06-5983-4924-94e2-5bb71856d733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:46.632701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1900f06-5983-4924-94e2-5bb71856d733 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:46.641300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:46.644404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 48/97] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:40:46 2022] GET /identity/v3/role_assignments?user.id=5109071b2f5d4a8fbd1639a297f22e81&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=b7f17471d2c948efa07d02424da68bea => generated 260 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:47.692620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f7ffd-5073-484c-aab0-a8325c03cad7 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:47.693682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f7ffd-5073-484c-aab0-a8325c03cad7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:47.694160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f7ffd-5073-484c-aab0-a8325c03cad7 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:47.695732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 50/98] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:47.708478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b788d65-2776-4b88-ab4a-9e4bd3f989c0 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:47.708941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b788d65-2776-4b88-ab4a-9e4bd3f989c0 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:47.709038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b788d65-2776-4b88-ab4a-9e4bd3f989c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:47.731125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7b788d65-2776-4b88-ab4a-9e4bd3f989c0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:47.733448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7b788d65-2776-4b88-ab4a-9e4bd3f989c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:47.821057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece0ea-ed10-48a5-b59f-2d16bc6a42fb None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:47.821674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece0ea-ed10-48a5-b59f-2d16bc6a42fb None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:47.822409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffece0ea-ed10-48a5-b59f-2d16bc6a42fb None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:47.824382 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 51/99] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:47 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:47.835675 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7befb5-68b6-4a53-836e-1e4e135b50e4 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:47.836229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7befb5-68b6-4a53-836e-1e4e135b50e4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:47.836693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7befb5-68b6-4a53-836e-1e4e135b50e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:47.853069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 49/100] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:47.869521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-1e7befb5-68b6-4a53-836e-1e4e135b50e4 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:47.871048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1e7befb5-68b6-4a53-836e-1e4e135b50e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:47.921820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:47.963166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7336b-60d1-4c01-b380-911e009e4c18 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:47.963750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7336b-60d1-4c01-b380-911e009e4c18 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:47.964240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f7336b-60d1-4c01-b380-911e009e4c18 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:47.965774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 50/101] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:40:47.975903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbd58f-3e3d-4f0d-9bf0-3f91e5432ad4 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:47.976161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbd58f-3e3d-4f0d-9bf0-3f91e5432ad4 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:47.976435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbd58f-3e3d-4f0d-9bf0-3f91e5432ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:47.985231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 52/102] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:48.000040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-57dbd58f-3e3d-4f0d-9bf0-3f91e5432ad4 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:48.001337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-57dbd58f-3e3d-4f0d-9bf0-3f91e5432ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:48.073276 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 51/103] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:40:48.360307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7befb5-68b6-4a53-836e-1e4e135b50e4 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:48.468193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbd58f-3e3d-4f0d-9bf0-3f91e5432ad4 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:48.479733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7befb5-68b6-4a53-836e-1e4e135b50e4 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:48.529617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:48.534513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49d11d-905b-4ee3-b11d-651a51ebf3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:48.535199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49d11d-905b-4ee3-b11d-651a51ebf3b8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:48.535485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49d11d-905b-4ee3-b11d-651a51ebf3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:48.535763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49d11d-905b-4ee3-b11d-651a51ebf3b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:48.551947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49d11d-905b-4ee3-b11d-651a51ebf3b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-5c49d11d-905b-4ee3-b11d-651a51ebf3b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:48.566466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:48.567499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 53/104] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:48.568822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49d11d-905b-4ee3-b11d-651a51ebf3b8 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:48.579678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbd58f-3e3d-4f0d-9bf0-3f91e5432ad4 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:48.616210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66312d86-4b75-4a0d-829d-7e3a3b42c8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:48.616783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66312d86-4b75-4a0d-829d-7e3a3b42c8b6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:48.617128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66312d86-4b75-4a0d-829d-7e3a3b42c8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:48.617402 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66312d86-4b75-4a0d-829d-7e3a3b42c8b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:48.618627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66312d86-4b75-4a0d-829d-7e3a3b42c8b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:48.620186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66312d86-4b75-4a0d-829d-7e3a3b42c8b6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:48.621986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 52/105] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:48 2022] GET /identity/v3/domains/default => generated 200 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:48.622792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66312d86-4b75-4a0d-829d-7e3a3b42c8b6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:48.633588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:48.741252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49d11d-905b-4ee3-b11d-651a51ebf3b8 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:48.788696 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153f30-8d33-469e-92ee-feed3c3b46fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:48.789339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153f30-8d33-469e-92ee-feed3c3b46fa None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:48.789623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153f30-8d33-469e-92ee-feed3c3b46fa None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:48.789895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3153f30-8d33-469e-92ee-feed3c3b46fa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:48.812076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3153f30-8d33-469e-92ee-feed3c3b46fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-d3153f30-8d33-469e-92ee-feed3c3b46fa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:48.816843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:48.817719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 54/106] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:48.818868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153f30-8d33-469e-92ee-feed3c3b46fa None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:48.858211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66312d86-4b75-4a0d-829d-7e3a3b42c8b6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:48.901746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:48.912302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5f453-7738-4a93-b393-74d22ae71c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:48.916860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5f453-7738-4a93-b393-74d22ae71c7f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:48.916860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5f453-7738-4a93-b393-74d22ae71c7f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:48.916860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee5f453-7738-4a93-b393-74d22ae71c7f None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:48.916860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5f453-7738-4a93-b393-74d22ae71c7f None admin] RBAC: Adding query filter params (name=member) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:48.916860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5f453-7738-4a93-b393-74d22ae71c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:48.916860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee5f453-7738-4a93-b393-74d22ae71c7f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:48.940106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 53/107] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:40:48 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:48.940746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5f453-7738-4a93-b393-74d22ae71c7f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:48.986901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:49.026201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3153f30-8d33-469e-92ee-feed3c3b46fa None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:49.096252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f9201-d884-4a23-be8e-e1298f643efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:49.097235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f9201-d884-4a23-be8e-e1298f643efb None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.097715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f9201-d884-4a23-be8e-e1298f643efb None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.098184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f9201-d884-4a23-be8e-e1298f643efb None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.099091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f9201-d884-4a23-be8e-e1298f643efb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:49.101491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f9201-d884-4a23-be8e-e1298f643efb None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:49.130185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee5f453-7738-4a93-b393-74d22ae71c7f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:49.166939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b95323-22f4-420a-846d-40d5bdaddf92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:49.167511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b95323-22f4-420a-846d-40d5bdaddf92 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.167797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b95323-22f4-420a-846d-40d5bdaddf92 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.168125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b95323-22f4-420a-846d-40d5bdaddf92 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.168799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b95323-22f4-420a-846d-40d5bdaddf92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:49.169126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b95323-22f4-420a-846d-40d5bdaddf92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:49.171734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b95323-22f4-420a-846d-40d5bdaddf92 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:49.181232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 54/108] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:40:48 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:49.185895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b95323-22f4-420a-846d-40d5bdaddf92 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:49.201346 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 55/109] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:40:48 2022] POST /identity/v3/projects => generated 303 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:40:49.203786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d834752-8bb8-4047-af94-a8b4079bba0c None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.204282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d834752-8bb8-4047-af94-a8b4079bba0c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.204786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d834752-8bb8-4047-af94-a8b4079bba0c None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.206448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 56/110] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:49 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:49.208221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d834752-8bb8-4047-af94-a8b4079bba0c None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:49.315856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:49.387947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b95323-22f4-420a-846d-40d5bdaddf92 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:49.428874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d834752-8bb8-4047-af94-a8b4079bba0c None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:49.431753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606be36-43fb-410f-adaf-ef6513e3d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:49.432620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606be36-43fb-410f-adaf-ef6513e3d635 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.433159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606be36-43fb-410f-adaf-ef6513e3d635 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.433629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606be36-43fb-410f-adaf-ef6513e3d635 None admin] PATH_INFO: `/v3/users/5109071b2f5d4a8fbd1639a297f22e81` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.435043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606be36-43fb-410f-adaf-ef6513e3d635 None admin] RBAC: Authorizing `identity:get_user(user_id=5109071b2f5d4a8fbd1639a297f22e81)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:49.437430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606be36-43fb-410f-adaf-ef6513e3d635 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:49.439470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 55/111] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:49 2022] GET /identity/v3/users/5109071b2f5d4a8fbd1639a297f22e81 => generated 276 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:49.441156 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bce40-5891-46f0-99c9-c260a2d82cb5 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.441640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bce40-5891-46f0-99c9-c260a2d82cb5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.442106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bce40-5891-46f0-99c9-c260a2d82cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.462612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:49.466772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-f21bce40-5891-46f0-99c9-c260a2d82cb5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:49.467628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e24fb-e238-413d-970a-8d2f51b57f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:49.468823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e24fb-e238-413d-970a-8d2f51b57f84 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.469106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e24fb-e238-413d-970a-8d2f51b57f84 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.469469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e24fb-e238-413d-970a-8d2f51b57f84 None admin] PATH_INFO: `/v3/users/ef3335126e8640d08c45ab4b702f3891` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.470301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f21bce40-5891-46f0-99c9-c260a2d82cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:49.484934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e24fb-e238-413d-970a-8d2f51b57f84 None admin] RBAC: Authorizing `identity:get_user(user_id=ef3335126e8640d08c45ab4b702f3891)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:49.486465 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e24fb-e238-413d-970a-8d2f51b57f84 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:49.491065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 57/112] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:49 2022] GET /identity/v3/users/ef3335126e8640d08c45ab4b702f3891 => generated 290 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:49.491065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e24fb-e238-413d-970a-8d2f51b57f84 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:49.566841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 56/113] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:49.572024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bce40-5891-46f0-99c9-c260a2d82cb5 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:49.611290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e24fb-e238-413d-970a-8d2f51b57f84 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:49.641027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:49.655350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76fed9-5b84-4871-b5b3-51eebb6a4f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:49.656559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76fed9-5b84-4871-b5b3-51eebb6a4f9c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.657373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76fed9-5b84-4871-b5b3-51eebb6a4f9c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.658139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76fed9-5b84-4871-b5b3-51eebb6a4f9c None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.659520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76fed9-5b84-4871-b5b3-51eebb6a4f9c None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:49.661360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76fed9-5b84-4871-b5b3-51eebb6a4f9c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:49.664328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 58/114] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:49 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:49.665685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8b48-8389-4655-b85c-e7e7affa871b None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.666164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8b48-8389-4655-b85c-e7e7affa871b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.666641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd8b48-8389-4655-b85c-e7e7affa871b None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.668061 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 59/115] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:49.673294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ae1e-713b-4f2f-892d-00bf9f9c3f60 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.674022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ae1e-713b-4f2f-892d-00bf9f9c3f60 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.674495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0ae1e-713b-4f2f-892d-00bf9f9c3f60 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.676891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 60/116] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:49 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:49.677824 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0ae1e-713b-4f2f-892d-00bf9f9c3f60 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:49.723990 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bce40-5891-46f0-99c9-c260a2d82cb5 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:49.751925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:49.760547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdaad95-3d74-455c-a963-3f21e5ef1450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:49.761173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaad95-3d74-455c-a963-3f21e5ef1450 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.761918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaad95-3d74-455c-a963-3f21e5ef1450 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.761918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdaad95-3d74-455c-a963-3f21e5ef1450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.798760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0ae1e-713b-4f2f-892d-00bf9f9c3f60 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:49.836988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6814b46c-101a-41dd-a6d8-957e0f516b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:49.837862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814b46c-101a-41dd-a6d8-957e0f516b16 None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.837986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaad95-3d74-455c-a963-3f21e5ef1450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:49.838191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814b46c-101a-41dd-a6d8-957e0f516b16 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.838409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6814b46c-101a-41dd-a6d8-957e0f516b16 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737/users/5109071b2f5d4a8fbd1639a297f22e81/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.839435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdaad95-3d74-455c-a963-3f21e5ef1450 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:49.840423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814b46c-101a-41dd-a6d8-957e0f516b16 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1f41b226cc94cccbb39608d62cb6737, user_id=5109071b2f5d4a8fbd1639a297f22e81, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:49.841956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6814b46c-101a-41dd-a6d8-957e0f516b16 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:49.888859 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:49.889387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 61/117] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:40:49 2022] PUT /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737/users/5109071b2f5d4a8fbd1639a297f22e81/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:40:49.901332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f2a68-a97a-4aed-bd1e-04f2d6beb875 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.902099 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f2a68-a97a-4aed-bd1e-04f2d6beb875 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.902986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f2a68-a97a-4aed-bd1e-04f2d6beb875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.927907 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-239f2a68-a97a-4aed-bd1e-04f2d6beb875 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:49.929759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-239f2a68-a97a-4aed-bd1e-04f2d6beb875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:49.957772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 57/118] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:49 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:40:49.959292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5c648-db7d-42a4-93ff-0fab1fab8533 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:49.959588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5c648-db7d-42a4-93ff-0fab1fab8533 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:49.959868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5c648-db7d-42a4-93ff-0fab1fab8533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:49.981505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-28d5c648-db7d-42a4-93ff-0fab1fab8533 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:49.982820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-28d5c648-db7d-42a4-93ff-0fab1fab8533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:50.023893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 62/119] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:50.025062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f2a68-a97a-4aed-bd1e-04f2d6beb875 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:50.092178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:50.092445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 58/120] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:49 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:50.095219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5c648-db7d-42a4-93ff-0fab1fab8533 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:50.197113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f2a68-a97a-4aed-bd1e-04f2d6beb875 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:50.244768 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:50.256900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c17f5-365d-434f-93e1-57e4d1103d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:50.264530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c17f5-365d-434f-93e1-57e4d1103d7f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:50.264530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c17f5-365d-434f-93e1-57e4d1103d7f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:50.264530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c17f5-365d-434f-93e1-57e4d1103d7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:50.266668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5c648-db7d-42a4-93ff-0fab1fab8533 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:50.278121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c17f5-365d-434f-93e1-57e4d1103d7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-c45c17f5-365d-434f-93e1-57e4d1103d7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:50.283605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:50.284644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:50.285822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:50.289849 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 63/121] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:40:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:50.291592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c17f5-365d-434f-93e1-57e4d1103d7f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:50.307649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125c77b-1b59-437d-9f0d-92d0045a2780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:50.308513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125c77b-1b59-437d-9f0d-92d0045a2780 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:50.308963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125c77b-1b59-437d-9f0d-92d0045a2780 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:50.309375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125c77b-1b59-437d-9f0d-92d0045a2780 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:50.310835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125c77b-1b59-437d-9f0d-92d0045a2780 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:50.313073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d125c77b-1b59-437d-9f0d-92d0045a2780 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:50.315045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 59/122] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:50 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:50.322998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125c77b-1b59-437d-9f0d-92d0045a2780 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:50.462673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125c77b-1b59-437d-9f0d-92d0045a2780 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:50.473274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c17f5-365d-434f-93e1-57e4d1103d7f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:50.486743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:50.501226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbf5a9-6723-472b-b476-4954229bd392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:50.502442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbf5a9-6723-472b-b476-4954229bd392 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:50.503148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbf5a9-6723-472b-b476-4954229bd392 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:50.503562 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbf5a9-6723-472b-b476-4954229bd392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:50.504602 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbf5a9-6723-472b-b476-4954229bd392 None admin] RBAC: Adding query filter params (role.id=0f01d78323f24797b5b1f2c555ea4c4c, scope.project.id=a1f41b226cc94cccbb39608d62cb6737, user.id=ef3335126e8640d08c45ab4b702f3891) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:50.505368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbf5a9-6723-472b-b476-4954229bd392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:50.507378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fbf5a9-6723-472b-b476-4954229bd392 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:50.514875 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 60/123] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:40:50 2022] GET /identity/v3/role_assignments?user.id=ef3335126e8640d08c45ab4b702f3891&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=0f01d78323f24797b5b1f2c555ea4c4c => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:50.516344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbf5a9-6723-472b-b476-4954229bd392 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:50.517062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9bffb-abf8-4393-8619-3f9a61f57681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:50.517869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9bffb-abf8-4393-8619-3f9a61f57681 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:50.518164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9bffb-abf8-4393-8619-3f9a61f57681 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:50.518450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9bffb-abf8-4393-8619-3f9a61f57681 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:50.519186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9bffb-abf8-4393-8619-3f9a61f57681 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:50.519499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9bffb-abf8-4393-8619-3f9a61f57681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:50.522333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9bffb-abf8-4393-8619-3f9a61f57681 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:50.534812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 64/124] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:50 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:50.535720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9bffb-abf8-4393-8619-3f9a61f57681 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:50.551456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:50.715803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbf5a9-6723-472b-b476-4954229bd392 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:50.733637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9bffb-abf8-4393-8619-3f9a61f57681 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:50.773774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8eca6e-14ee-44dd-905a-645116b63a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:50.774355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eca6e-14ee-44dd-905a-645116b63a5f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:50.774594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eca6e-14ee-44dd-905a-645116b63a5f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:50.774865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8eca6e-14ee-44dd-905a-645116b63a5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:50.786437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8eca6e-14ee-44dd-905a-645116b63a5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-bb8eca6e-14ee-44dd-905a-645116b63a5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:50.792182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:50.793035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:50.793794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 65/125] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:50.794416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8eca6e-14ee-44dd-905a-645116b63a5f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:50.799736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606d446-f951-4a0d-8a1a-ab7b671e475d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:50.800671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606d446-f951-4a0d-8a1a-ab7b671e475d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:50.801145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606d446-f951-4a0d-8a1a-ab7b671e475d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:50.801601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606d446-f951-4a0d-8a1a-ab7b671e475d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:50.827720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606d446-f951-4a0d-8a1a-ab7b671e475d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-f606d446-f951-4a0d-8a1a-ab7b671e475d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:50.834388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:50.835273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:50.836061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 61/126] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:40:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:50.836698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606d446-f951-4a0d-8a1a-ab7b671e475d None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:50.843098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:50.915416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:50.943742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606d446-f951-4a0d-8a1a-ab7b671e475d None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:50.979229 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67d22b-29fb-4d27-beaf-e5fa3731880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:50.979842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67d22b-29fb-4d27-beaf-e5fa3731880e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:50.980119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67d22b-29fb-4d27-beaf-e5fa3731880e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:50.980388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67d22b-29fb-4d27-beaf-e5fa3731880e None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:50.981048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67d22b-29fb-4d27-beaf-e5fa3731880e None admin] RBAC: Adding query filter params (name=member) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:50.981347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67d22b-29fb-4d27-beaf-e5fa3731880e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:50.982858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d67d22b-29fb-4d27-beaf-e5fa3731880e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:50.991685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 62/127] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:40:50 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:50.992513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67d22b-29fb-4d27-beaf-e5fa3731880e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:51.084183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8eca6e-14ee-44dd-905a-645116b63a5f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:51.127751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df61f01-209f-49f5-a607-44feb2d80a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:51.128405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df61f01-209f-49f5-a607-44feb2d80a18 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:51.128693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df61f01-209f-49f5-a607-44feb2d80a18 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:51.129115 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df61f01-209f-49f5-a607-44feb2d80a18 None admin] PATH_INFO: `/v3/users/f6a4ad480e8041b39ccfa602ac02988c` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:51.130323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df61f01-209f-49f5-a607-44feb2d80a18 None admin] RBAC: Authorizing `identity:get_user(user_id=f6a4ad480e8041b39ccfa602ac02988c)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:51.131818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df61f01-209f-49f5-a607-44feb2d80a18 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:51.133400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 66/128] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:50 2022] GET /identity/v3/users/f6a4ad480e8041b39ccfa602ac02988c => generated 248 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:51.134250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df61f01-209f-49f5-a607-44feb2d80a18 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:51.150088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67d22b-29fb-4d27-beaf-e5fa3731880e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:51.215804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9938b-9684-42b5-b2b6-92c9b5eb1d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:51.216436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9938b-9684-42b5-b2b6-92c9b5eb1d32 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:51.216719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9938b-9684-42b5-b2b6-92c9b5eb1d32 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:51.217041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9938b-9684-42b5-b2b6-92c9b5eb1d32 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:51.217657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9938b-9684-42b5-b2b6-92c9b5eb1d32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:51.217963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9938b-9684-42b5-b2b6-92c9b5eb1d32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:51.223743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c9938b-9684-42b5-b2b6-92c9b5eb1d32 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:51.241820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:51.243346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 63/129] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:40:50 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:51.244007 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9938b-9684-42b5-b2b6-92c9b5eb1d32 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:51.253694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:51.257639 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df61f01-209f-49f5-a607-44feb2d80a18 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:51.295425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317de984-5db5-4129-b475-5bdd14e383d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:51.297393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317de984-5db5-4129-b475-5bdd14e383d1 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:51.297763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317de984-5db5-4129-b475-5bdd14e383d1 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:51.298141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317de984-5db5-4129-b475-5bdd14e383d1 None admin] PATH_INFO: `/v3/users/5109071b2f5d4a8fbd1639a297f22e81` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:51.301346 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317de984-5db5-4129-b475-5bdd14e383d1 None admin] RBAC: Authorizing `identity:get_user(user_id=5109071b2f5d4a8fbd1639a297f22e81)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:51.304500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317de984-5db5-4129-b475-5bdd14e383d1 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:51.308175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 67/130] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:51 2022] GET /identity/v3/users/5109071b2f5d4a8fbd1639a297f22e81 => generated 276 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:51.309745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317de984-5db5-4129-b475-5bdd14e383d1 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:51.434543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317de984-5db5-4129-b475-5bdd14e383d1 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:51.469482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c09c7-cb8b-4f44-ad5b-f2345cd4d07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:51.470153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c09c7-cb8b-4f44-ad5b-f2345cd4d07d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:51.470428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c09c7-cb8b-4f44-ad5b-f2345cd4d07d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:51.470699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c09c7-cb8b-4f44-ad5b-f2345cd4d07d None admin] PATH_INFO: `/v3/users/5109071b2f5d4a8fbd1639a297f22e81` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:51.471875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c09c7-cb8b-4f44-ad5b-f2345cd4d07d None admin] RBAC: Authorizing `identity:get_user(user_id=5109071b2f5d4a8fbd1639a297f22e81)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:51.473408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c09c7-cb8b-4f44-ad5b-f2345cd4d07d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:51.475006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 68/131] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:51 2022] GET /identity/v3/users/5109071b2f5d4a8fbd1639a297f22e81 => generated 276 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:51.479985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c09c7-cb8b-4f44-ad5b-f2345cd4d07d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:51.523434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:51.554413 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9938b-9684-42b5-b2b6-92c9b5eb1d32 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:51.587110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c09c7-cb8b-4f44-ad5b-f2345cd4d07d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:51.624478 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048f439-b9c6-478f-8208-7dd2db3de5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:51.625306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048f439-b9c6-478f-8208-7dd2db3de5b2 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:51.625626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048f439-b9c6-478f-8208-7dd2db3de5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:51.625906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9048f439-b9c6-478f-8208-7dd2db3de5b2 None admin] PATH_INFO: `/v3/projects/7cefd8a371b74c568801146b5a72469b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:51.627642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048f439-b9c6-478f-8208-7dd2db3de5b2 None admin] RBAC: Authorizing `identity:get_project(project_id=7cefd8a371b74c568801146b5a72469b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:51.628637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a53b68-3546-4dab-8418-54ac7e28f679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:51.629230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9048f439-b9c6-478f-8208-7dd2db3de5b2 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:51.629334 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a53b68-3546-4dab-8418-54ac7e28f679 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:51.629559 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a53b68-3546-4dab-8418-54ac7e28f679 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:51.629839 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a53b68-3546-4dab-8418-54ac7e28f679 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:51.630972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 69/132] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:51 2022] GET /identity/v3/projects/7cefd8a371b74c568801146b5a72469b => generated 313 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:51.631863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a53b68-3546-4dab-8418-54ac7e28f679 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:51.632704 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048f439-b9c6-478f-8208-7dd2db3de5b2 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:51.634956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a53b68-3546-4dab-8418-54ac7e28f679 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:51.639523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 64/133] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:51 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:51.642883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a53b68-3546-4dab-8418-54ac7e28f679 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:51.719974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:51.748687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9048f439-b9c6-478f-8208-7dd2db3de5b2 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:51.750377 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:51.772422 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a53b68-3546-4dab-8418-54ac7e28f679 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:51.786192 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029a428-d356-477b-96e6-e033f7454a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:51.787225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029a428-d356-477b-96e6-e033f7454a21 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:51.787578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029a428-d356-477b-96e6-e033f7454a21 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:51.787930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029a428-d356-477b-96e6-e033f7454a21 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:51.789648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029a428-d356-477b-96e6-e033f7454a21 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:51.791215 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029a428-d356-477b-96e6-e033f7454a21 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:51.792937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 70/134] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:51 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:51.799530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e642b-3a01-4915-864d-d324d0a65a1f None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:51.800000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e642b-3a01-4915-864d-d324d0a65a1f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:51.800344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e642b-3a01-4915-864d-d324d0a65a1f None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:51.801681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 71/135] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:51 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:51.802678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e642b-3a01-4915-864d-d324d0a65a1f None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:51.820373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee9d26-86ce-4098-9080-f679c8a73795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:51.821051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9d26-86ce-4098-9080-f679c8a73795 None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:51.821650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9d26-86ce-4098-9080-f679c8a73795 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:51.821929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9d26-86ce-4098-9080-f679c8a73795 None admin] PATH_INFO: `/v3/projects/7cefd8a371b74c568801146b5a72469b/users/5109071b2f5d4a8fbd1639a297f22e81/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:51.824995 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee9d26-86ce-4098-9080-f679c8a73795 None admin] RBAC: Authorizing `identity:create_grant(project_id=7cefd8a371b74c568801146b5a72469b, user_id=5109071b2f5d4a8fbd1639a297f22e81, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:51.827341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee9d26-86ce-4098-9080-f679c8a73795 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:51.919213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e642b-3a01-4915-864d-d324d0a65a1f None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:51.933068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 65/136] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:40:51 2022] PUT /identity/v3/projects/7cefd8a371b74c568801146b5a72469b/users/5109071b2f5d4a8fbd1639a297f22e81/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:40:51.933748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee9d26-86ce-4098-9080-f679c8a73795 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:51.992498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88191ded-8606-4d68-b85d-5c6b6b46d6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:51.993090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191ded-8606-4d68-b85d-5c6b6b46d6f8 None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:51.993371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191ded-8606-4d68-b85d-5c6b6b46d6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:51.993645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88191ded-8606-4d68-b85d-5c6b6b46d6f8 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737/users/f6a4ad480e8041b39ccfa602ac02988c/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:51.995801 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88191ded-8606-4d68-b85d-5c6b6b46d6f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1f41b226cc94cccbb39608d62cb6737, user_id=f6a4ad480e8041b39ccfa602ac02988c, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:51.997291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88191ded-8606-4d68-b85d-5c6b6b46d6f8 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:52.089994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee9d26-86ce-4098-9080-f679c8a73795 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:52.107926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 72/137] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:40:51 2022] PUT /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737/users/f6a4ad480e8041b39ccfa602ac02988c/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:40:52.109375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60bb6b-aca6-4bfb-82f4-da65f032f0c6 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:52.109666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60bb6b-aca6-4bfb-82f4-da65f032f0c6 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:52.109942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60bb6b-aca6-4bfb-82f4-da65f032f0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:52.146367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:52.146708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-6d60bb6b-aca6-4bfb-82f4-da65f032f0c6 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:52.148754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6d60bb6b-aca6-4bfb-82f4-da65f032f0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:52.176044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dcf3e-c03d-41fb-b312-21b7f8e055fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:52.176044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dcf3e-c03d-41fb-b312-21b7f8e055fd None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:52.176044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dcf3e-c03d-41fb-b312-21b7f8e055fd None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:52.176044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dcf3e-c03d-41fb-b312-21b7f8e055fd None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737/users/5109071b2f5d4a8fbd1639a297f22e81/roles/b7f17471d2c948efa07d02424da68bea` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:52.183061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dcf3e-c03d-41fb-b312-21b7f8e055fd None admin] RBAC: Authorizing `identity:create_grant(project_id=a1f41b226cc94cccbb39608d62cb6737, user_id=5109071b2f5d4a8fbd1639a297f22e81, role_id=b7f17471d2c948efa07d02424da68bea)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:52.183061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dcf3e-c03d-41fb-b312-21b7f8e055fd None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:52.199947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:52.257549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 66/138] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:40:51 2022] PUT /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737/users/5109071b2f5d4a8fbd1639a297f22e81/roles/b7f17471d2c948efa07d02424da68bea => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:40:52.271839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 73/139] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:52.292497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce44530-8746-4a2e-8ad8-1094f44c5fdb None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:52.293146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce44530-8746-4a2e-8ad8-1094f44c5fdb None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:52.293429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce44530-8746-4a2e-8ad8-1094f44c5fdb None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:52.294974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 67/140] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:52.301965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4944b0-b947-40ac-af8e-9f1c495db264 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:52.302292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4944b0-b947-40ac-af8e-9f1c495db264 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:52.302530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4944b0-b947-40ac-af8e-9f1c495db264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:52.324138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-aa4944b0-b947-40ac-af8e-9f1c495db264 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:52.325375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-aa4944b0-b947-40ac-af8e-9f1c495db264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:52.425241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 74/141] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:52.650738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce44530-8746-4a2e-8ad8-1094f44c5fdb None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:52.742694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d5272-dab6-45ed-a213-75682da6ab33 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:52.743011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d5272-dab6-45ed-a213-75682da6ab33 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:52.743311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d5272-dab6-45ed-a213-75682da6ab33 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:52.744742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 75/142] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:52.749842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ead41-ee03-4d66-9133-c28e5d6bcdc7 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:52.750125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ead41-ee03-4d66-9133-c28e5d6bcdc7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:52.750399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ead41-ee03-4d66-9133-c28e5d6bcdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:52.764174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce44530-8746-4a2e-8ad8-1094f44c5fdb None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:52.782529 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-206ead41-ee03-4d66-9133-c28e5d6bcdc7 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:52.783906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-206ead41-ee03-4d66-9133-c28e5d6bcdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:52.806877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884869c3-4d61-4b15-9159-004f6c52dc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:52.807883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884869c3-4d61-4b15-9159-004f6c52dc0b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:52.808565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884869c3-4d61-4b15-9159-004f6c52dc0b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:52.809524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884869c3-4d61-4b15-9159-004f6c52dc0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:52.810979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884869c3-4d61-4b15-9159-004f6c52dc0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:52.813096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884869c3-4d61-4b15-9159-004f6c52dc0b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:52.816163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 68/143] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:52 2022] GET /identity/v3/domains/default => generated 200 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:52.822496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884869c3-4d61-4b15-9159-004f6c52dc0b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:52.888665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:52.911629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 76/144] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:40:52.929668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ead41-ee03-4d66-9133-c28e5d6bcdc7 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:52.947116 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884869c3-4d61-4b15-9159-004f6c52dc0b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:52.959447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:52.991455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b105a-5c48-4b8f-aa74-48e8b91cf8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:52.992109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b105a-5c48-4b8f-aa74-48e8b91cf8a6 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:52.992440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b105a-5c48-4b8f-aa74-48e8b91cf8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:52.992721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b105a-5c48-4b8f-aa74-48e8b91cf8a6 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:52.993373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b105a-5c48-4b8f-aa74-48e8b91cf8a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:52.994826 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b105a-5c48-4b8f-aa74-48e8b91cf8a6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:53.057786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ead41-ee03-4d66-9133-c28e5d6bcdc7 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:53.096719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d14407-a247-4591-8e78-03c1e96b8807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:53.097741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d14407-a247-4591-8e78-03c1e96b8807 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:53.098238 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d14407-a247-4591-8e78-03c1e96b8807 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:53.098709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d14407-a247-4591-8e78-03c1e96b8807 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:53.110627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d14407-a247-4591-8e78-03c1e96b8807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-51d14407-a247-4591-8e78-03c1e96b8807 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:53.116069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:53.116918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:53.117716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:53.118295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 69/145] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:40:52 2022] POST /identity/v3/users => generated 284 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:40:53.120434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 77/146] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:40:53.126590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b105a-5c48-4b8f-aa74-48e8b91cf8a6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:53.289242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b105a-5c48-4b8f-aa74-48e8b91cf8a6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:53.299764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:53.326263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb35be9-bc7f-4225-a36b-8c9077941d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:53.326881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb35be9-bc7f-4225-a36b-8c9077941d99 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:53.327194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb35be9-bc7f-4225-a36b-8c9077941d99 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:53.327470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb35be9-bc7f-4225-a36b-8c9077941d99 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:53.328138 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb35be9-bc7f-4225-a36b-8c9077941d99 None admin] RBAC: Adding query filter params (name=member) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:53.328455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb35be9-bc7f-4225-a36b-8c9077941d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:53.329932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb35be9-bc7f-4225-a36b-8c9077941d99 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:53.345697 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 70/147] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:40:53 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:53.347845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d14407-a247-4591-8e78-03c1e96b8807 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:53.402292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb35be9-bc7f-4225-a36b-8c9077941d99 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:53.481601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:53.519175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d14407-a247-4591-8e78-03c1e96b8807 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:53.574379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d792c85-cbfa-48f5-aa38-55d79b271525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:53.575062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d792c85-cbfa-48f5-aa38-55d79b271525 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:53.575311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d792c85-cbfa-48f5-aa38-55d79b271525 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:53.575579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d792c85-cbfa-48f5-aa38-55d79b271525 None admin] PATH_INFO: `/v3/users/5109071b2f5d4a8fbd1639a297f22e81` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:53.577371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d792c85-cbfa-48f5-aa38-55d79b271525 None admin] RBAC: Authorizing `identity:get_user(user_id=5109071b2f5d4a8fbd1639a297f22e81)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:53.578845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d792c85-cbfa-48f5-aa38-55d79b271525 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:53.580608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 78/148] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:53 2022] GET /identity/v3/users/5109071b2f5d4a8fbd1639a297f22e81 => generated 276 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:53.585923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb35be9-bc7f-4225-a36b-8c9077941d99 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:53.591902 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d792c85-cbfa-48f5-aa38-55d79b271525 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:53.654512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ced6a-765b-48a9-afe1-2afca0f19b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:53.655169 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ced6a-765b-48a9-afe1-2afca0f19b00 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:53.655502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ced6a-765b-48a9-afe1-2afca0f19b00 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:53.655772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ced6a-765b-48a9-afe1-2afca0f19b00 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:53.677713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58ced6a-765b-48a9-afe1-2afca0f19b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-c58ced6a-765b-48a9-afe1-2afca0f19b00 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:53.683495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:53.684353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:40:53.685213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:53.687013 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 71/149] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:40:53.693998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ced6a-765b-48a9-afe1-2afca0f19b00 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:53.750306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:53.789584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d792c85-cbfa-48f5-aa38-55d79b271525 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:53.855107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f41102-0a41-4444-86d6-2931de5614b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:53.855729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f41102-0a41-4444-86d6-2931de5614b2 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:53.856110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f41102-0a41-4444-86d6-2931de5614b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:53.856387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f41102-0a41-4444-86d6-2931de5614b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:53.878221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ced6a-765b-48a9-afe1-2afca0f19b00 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:53.899911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:53.953883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da048cd6-05ba-4f8e-8a40-d901568b6a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:53.954537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da048cd6-05ba-4f8e-8a40-d901568b6a8f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:53.954763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da048cd6-05ba-4f8e-8a40-d901568b6a8f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:53.955054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da048cd6-05ba-4f8e-8a40-d901568b6a8f None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:53.955729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da048cd6-05ba-4f8e-8a40-d901568b6a8f None admin] RBAC: Adding query filter params (name=reader) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:53.956556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da048cd6-05ba-4f8e-8a40-d901568b6a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:53.961284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da048cd6-05ba-4f8e-8a40-d901568b6a8f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:53.966568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f41102-0a41-4444-86d6-2931de5614b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:53.968062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f41102-0a41-4444-86d6-2931de5614b2 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:53.981364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 72/150] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:40:53 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:53.987786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da048cd6-05ba-4f8e-8a40-d901568b6a8f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:54.121912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:54.132122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da048cd6-05ba-4f8e-8a40-d901568b6a8f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:54.147148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 79/151] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:53 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:40:54.154701 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f41102-0a41-4444-86d6-2931de5614b2 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:54.174667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec8049-8f64-40c2-8af7-9fb0e86876ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:54.176310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec8049-8f64-40c2-8af7-9fb0e86876ee None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:54.176681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec8049-8f64-40c2-8af7-9fb0e86876ee None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:54.176965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec8049-8f64-40c2-8af7-9fb0e86876ee None admin] PATH_INFO: `/v3/users/ef3335126e8640d08c45ab4b702f3891` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:54.183049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec8049-8f64-40c2-8af7-9fb0e86876ee None admin] RBAC: Authorizing `identity:get_user(user_id=ef3335126e8640d08c45ab4b702f3891)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:54.183049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec8049-8f64-40c2-8af7-9fb0e86876ee None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:54.183049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 73/152] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:53 2022] GET /identity/v3/users/ef3335126e8640d08c45ab4b702f3891 => generated 290 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:54.189367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec8049-8f64-40c2-8af7-9fb0e86876ee None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:54.232956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:54.309587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec8049-8f64-40c2-8af7-9fb0e86876ee None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:54.330045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f41102-0a41-4444-86d6-2931de5614b2 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:54.348526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab80739-a94a-4092-a636-5fa07dbf75cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:54.349288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab80739-a94a-4092-a636-5fa07dbf75cb None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:54.349575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab80739-a94a-4092-a636-5fa07dbf75cb None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:54.349949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab80739-a94a-4092-a636-5fa07dbf75cb None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:54.351132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab80739-a94a-4092-a636-5fa07dbf75cb None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:54.352713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab80739-a94a-4092-a636-5fa07dbf75cb None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:54.354706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 74/153] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:54 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:54.360596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab80739-a94a-4092-a636-5fa07dbf75cb None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:54.391871 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad407-4fb0-45b7-bae4-fb7bf1c019c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:54.392450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad407-4fb0-45b7-bae4-fb7bf1c019c4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:54.392731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad407-4fb0-45b7-bae4-fb7bf1c019c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:54.393008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad407-4fb0-45b7-bae4-fb7bf1c019c4 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:54.394282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad407-4fb0-45b7-bae4-fb7bf1c019c4 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:54.395925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad407-4fb0-45b7-bae4-fb7bf1c019c4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:54.397595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 80/154] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:54 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:54.408026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad407-4fb0-45b7-bae4-fb7bf1c019c4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:54.408668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:54.476256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab80739-a94a-4092-a636-5fa07dbf75cb None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:54.516400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e7761-222e-4d43-b111-ae60d2abb5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:54.517071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e7761-222e-4d43-b111-ae60d2abb5a3 None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:54.517350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e7761-222e-4d43-b111-ae60d2abb5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:54.517618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e7761-222e-4d43-b111-ae60d2abb5a3 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737/users/ef3335126e8640d08c45ab4b702f3891/roles/0f01d78323f24797b5b1f2c555ea4c4c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:54.519787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e7761-222e-4d43-b111-ae60d2abb5a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1f41b226cc94cccbb39608d62cb6737, user_id=ef3335126e8640d08c45ab4b702f3891, role_id=0f01d78323f24797b5b1f2c555ea4c4c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:54.521347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e7761-222e-4d43-b111-ae60d2abb5a3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:54.534930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad407-4fb0-45b7-bae4-fb7bf1c019c4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:54.552853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:54.572577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3eac7-4600-4db2-9a78-40f116d39aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:54.573192 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3eac7-4600-4db2-9a78-40f116d39aed None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:54.573471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3eac7-4600-4db2-9a78-40f116d39aed None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:54.573746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3eac7-4600-4db2-9a78-40f116d39aed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:54.574484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3eac7-4600-4db2-9a78-40f116d39aed None admin] RBAC: Adding query filter params (role.id=df27fa65390e4de78fa117773dc73555, scope.project.id=a1f41b226cc94cccbb39608d62cb6737, user.id=5109071b2f5d4a8fbd1639a297f22e81) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:54.574793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3eac7-4600-4db2-9a78-40f116d39aed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:54.576359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3eac7-4600-4db2-9a78-40f116d39aed None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:54.582617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 81/155] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:40:54 2022] GET /identity/v3/role_assignments?user.id=5109071b2f5d4a8fbd1639a297f22e81&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=df27fa65390e4de78fa117773dc73555 => generated 610 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:54.606159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 75/156] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:40:54 2022] PUT /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737/users/ef3335126e8640d08c45ab4b702f3891/roles/0f01d78323f24797b5b1f2c555ea4c4c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:40:55.005830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb54937-9707-49a1-bd34-908e4641fd3a None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:55.006431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb54937-9707-49a1-bd34-908e4641fd3a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:55.006431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb54937-9707-49a1-bd34-908e4641fd3a None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:55.008084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 82/157] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:55.013722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4626f4-6d00-4654-a5c7-dd244019aeff None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:55.014321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4626f4-6d00-4654-a5c7-dd244019aeff None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:55.014321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4626f4-6d00-4654-a5c7-dd244019aeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:55.040057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8a4626f4-6d00-4654-a5c7-dd244019aeff None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:55.041283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8a4626f4-6d00-4654-a5c7-dd244019aeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:55.117188 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 76/158] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:55.256514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53728576-ee4f-4dc3-9d95-60c432e9c463 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:55.256817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53728576-ee4f-4dc3-9d95-60c432e9c463 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:55.256984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53728576-ee4f-4dc3-9d95-60c432e9c463 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:55.258336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 83/159] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:55.273364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa7e0d-a956-40ad-a1a3-830c7c7c8467 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:55.273658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa7e0d-a956-40ad-a1a3-830c7c7c8467 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:55.274018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa7e0d-a956-40ad-a1a3-830c7c7c8467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:55.304106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e8fa7e0d-a956-40ad-a1a3-830c7c7c8467 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:55.305325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e8fa7e0d-a956-40ad-a1a3-830c7c7c8467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:55.320989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:55.386177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 77/160] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:55 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:55.451113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53728576-ee4f-4dc3-9d95-60c432e9c463 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:55.568921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53728576-ee4f-4dc3-9d95-60c432e9c463 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:55.615525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f704b84-4fe8-490b-b2f7-3bf8f3da5c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:55.616601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f704b84-4fe8-490b-b2f7-3bf8f3da5c3d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:55.617079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f704b84-4fe8-490b-b2f7-3bf8f3da5c3d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:55.617545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f704b84-4fe8-490b-b2f7-3bf8f3da5c3d None admin] PATH_INFO: `/v3/roles/member` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:55.628874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f704b84-4fe8-490b-b2f7-3bf8f3da5c3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-4f704b84-4fe8-490b-b2f7-3bf8f3da5c3d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:55.639640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:55.640591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:40:55.641451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:55.644067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 84/161] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:55.651129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa7e0d-a956-40ad-a1a3-830c7c7c8467 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:55.782420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa7e0d-a956-40ad-a1a3-830c7c7c8467 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:55.822142 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7d780-f8f7-42eb-853a-f94a2e84c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:55.822678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d780-f8f7-42eb-853a-f94a2e84c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:55.822954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d780-f8f7-42eb-853a-f94a2e84c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:55.823255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7d780-f8f7-42eb-853a-f94a2e84c4a2 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:55.823983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7d780-f8f7-42eb-853a-f94a2e84c4a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:55.824282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7d780-f8f7-42eb-853a-f94a2e84c4a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:55.825660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f7d780-f8f7-42eb-853a-f94a2e84c4a2 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:55.834689 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 78/162] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:40:55 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:55.840448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f704b84-4fe8-490b-b2f7-3bf8f3da5c3d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:55.884633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:55.931563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7d780-f8f7-42eb-853a-f94a2e84c4a2 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:55.947222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:55.986088 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f704b84-4fe8-490b-b2f7-3bf8f3da5c3d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:56.055501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a528dd-8bd8-4bbb-816b-ae42561d4522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:56.056146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a528dd-8bd8-4bbb-816b-ae42561d4522 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:56.058927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a528dd-8bd8-4bbb-816b-ae42561d4522 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:56.059309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a528dd-8bd8-4bbb-816b-ae42561d4522 None admin] PATH_INFO: `/v3/users/5109071b2f5d4a8fbd1639a297f22e81` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:56.061846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a528dd-8bd8-4bbb-816b-ae42561d4522 None admin] RBAC: Authorizing `identity:get_user(user_id=5109071b2f5d4a8fbd1639a297f22e81)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:56.064523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a528dd-8bd8-4bbb-816b-ae42561d4522 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:56.069234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 85/163] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:55 2022] GET /identity/v3/users/5109071b2f5d4a8fbd1639a297f22e81 => generated 276 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:56.104485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a528dd-8bd8-4bbb-816b-ae42561d4522 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:56.109000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7d780-f8f7-42eb-853a-f94a2e84c4a2 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:56.176256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516f6db-332d-4ad8-9856-dff347142e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:56.177629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516f6db-332d-4ad8-9856-dff347142e97 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:56.178063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516f6db-332d-4ad8-9856-dff347142e97 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:56.178341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516f6db-332d-4ad8-9856-dff347142e97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:56.209004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516f6db-332d-4ad8-9856-dff347142e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-2516f6db-332d-4ad8-9856-dff347142e97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:56.214173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:56.215063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:56.215974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:56.225117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 79/164] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:40:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:40:56.226382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd736-b133-48bf-8177-878a7228f620 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:56.226657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd736-b133-48bf-8177-878a7228f620 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:56.226975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dd736-b133-48bf-8177-878a7228f620 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:56.231593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 80/165] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:56 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:56.233231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dd736-b133-48bf-8177-878a7228f620 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:56.307304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a528dd-8bd8-4bbb-816b-ae42561d4522 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:56.316644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:56.343464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3c2ad3-9d13-4210-90d0-7a7de0d8b317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:56.344074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c2ad3-9d13-4210-90d0-7a7de0d8b317 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:56.344350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c2ad3-9d13-4210-90d0-7a7de0d8b317 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:56.344616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3c2ad3-9d13-4210-90d0-7a7de0d8b317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:56.409032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:56.418339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c2ad3-9d13-4210-90d0-7a7de0d8b317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:56.424925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3c2ad3-9d13-4210-90d0-7a7de0d8b317 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:56.496689 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dd736-b133-48bf-8177-878a7228f620 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:56.526382 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 86/166] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:56 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 04:40:56.527683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dcbc4-bfcb-4603-a1dd-e18bab60b14a None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:56.527963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dcbc4-bfcb-4603-a1dd-e18bab60b14a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:56.528294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dcbc4-bfcb-4603-a1dd-e18bab60b14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:56.547462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2b6dcbc4-bfcb-4603-a1dd-e18bab60b14a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:56.548636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2b6dcbc4-bfcb-4603-a1dd-e18bab60b14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:56.550629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:56.596270 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759e540-5ca9-4851-9506-af8b1e3c472a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:56.596903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759e540-5ca9-4851-9506-af8b1e3c472a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:56.597182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759e540-5ca9-4851-9506-af8b1e3c472a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:56.597520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759e540-5ca9-4851-9506-af8b1e3c472a None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:56.598125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759e540-5ca9-4851-9506-af8b1e3c472a None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:56.598428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759e540-5ca9-4851-9506-af8b1e3c472a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:56.600063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759e540-5ca9-4851-9506-af8b1e3c472a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:56.627048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 81/167] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:56 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:56.627048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759e540-5ca9-4851-9506-af8b1e3c472a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:56.631922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 87/168] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:56.632811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dcbc4-bfcb-4603-a1dd-e18bab60b14a None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:56.887181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759e540-5ca9-4851-9506-af8b1e3c472a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:56.890300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dcbc4-bfcb-4603-a1dd-e18bab60b14a None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:56.941127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:56.948779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba13f2a-1f25-49d7-93b5-d01292c451df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:56.949548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73194a5d-ac2d-470b-9c5b-f0295b2fecc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:56.949893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba13f2a-1f25-49d7-93b5-d01292c451df None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:56.950182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73194a5d-ac2d-470b-9c5b-f0295b2fecc0 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:56.950465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73194a5d-ac2d-470b-9c5b-f0295b2fecc0 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:56.950685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba13f2a-1f25-49d7-93b5-d01292c451df None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:56.950790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73194a5d-ac2d-470b-9c5b-f0295b2fecc0 None admin] PATH_INFO: `/v3/projects/7cefd8a371b74c568801146b5a72469b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:56.951245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba13f2a-1f25-49d7-93b5-d01292c451df None admin] PATH_INFO: `/v3/users/f6a4ad480e8041b39ccfa602ac02988c` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:56.960640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba13f2a-1f25-49d7-93b5-d01292c451df None admin] RBAC: Authorizing `identity:get_user(user_id=f6a4ad480e8041b39ccfa602ac02988c)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:56.960640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba13f2a-1f25-49d7-93b5-d01292c451df None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:56.960816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73194a5d-ac2d-470b-9c5b-f0295b2fecc0 None admin] RBAC: Authorizing `identity:get_project(project_id=7cefd8a371b74c568801146b5a72469b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:56.960816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73194a5d-ac2d-470b-9c5b-f0295b2fecc0 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:56.960816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 82/169] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:56 2022] GET /identity/v3/projects/7cefd8a371b74c568801146b5a72469b => generated 313 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:56.961009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 88/170] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:56 2022] GET /identity/v3/users/f6a4ad480e8041b39ccfa602ac02988c => generated 248 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:56.968002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba13f2a-1f25-49d7-93b5-d01292c451df None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:56.968151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73194a5d-ac2d-470b-9c5b-f0295b2fecc0 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:56.978191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:57.117699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba13f2a-1f25-49d7-93b5-d01292c451df None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:57.132122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73194a5d-ac2d-470b-9c5b-f0295b2fecc0 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:57.189217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e33ee-9a5c-4ff7-adc2-b2fe34458aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:57.189765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e33ee-9a5c-4ff7-adc2-b2fe34458aca None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:57.190659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e33ee-9a5c-4ff7-adc2-b2fe34458aca None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:57.190659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e33ee-9a5c-4ff7-adc2-b2fe34458aca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:57.191157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e33ee-9a5c-4ff7-adc2-b2fe34458aca None admin] RBAC: Adding query filter params (role.id=df27fa65390e4de78fa117773dc73555, scope.project.id=7cefd8a371b74c568801146b5a72469b, user.id=5109071b2f5d4a8fbd1639a297f22e81) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:57.191575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e33ee-9a5c-4ff7-adc2-b2fe34458aca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:57.193209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e33ee-9a5c-4ff7-adc2-b2fe34458aca None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:57.193704 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da6418-409e-46eb-80ba-9d41a08f6d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:57.194428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da6418-409e-46eb-80ba-9d41a08f6d0d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:57.194771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da6418-409e-46eb-80ba-9d41a08f6d0d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:57.195134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da6418-409e-46eb-80ba-9d41a08f6d0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:57.206164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 83/171] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:40:56 2022] GET /identity/v3/role_assignments?user.id=5109071b2f5d4a8fbd1639a297f22e81&scope.project.id=7cefd8a371b74c568801146b5a72469b&role.id=df27fa65390e4de78fa117773dc73555 => generated 610 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:57.207537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46368eb2-87db-4d5f-84c1-9f276e510c76 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:57.207737 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46368eb2-87db-4d5f-84c1-9f276e510c76 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:57.208018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46368eb2-87db-4d5f-84c1-9f276e510c76 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:57.209474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 84/172] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:57.210261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46368eb2-87db-4d5f-84c1-9f276e510c76 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:57.281315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da6418-409e-46eb-80ba-9d41a08f6d0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:57.283062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da6418-409e-46eb-80ba-9d41a08f6d0d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:57.298345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:57.303212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:57.373188 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46368eb2-87db-4d5f-84c1-9f276e510c76 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:57.428325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105c08e-952e-4915-814f-c9afb17f3ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:57.429713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105c08e-952e-4915-814f-c9afb17f3ce7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:57.430225 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105c08e-952e-4915-814f-c9afb17f3ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:57.431193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105c08e-952e-4915-814f-c9afb17f3ce7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:57.446551 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105c08e-952e-4915-814f-c9afb17f3ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-1105c08e-952e-4915-814f-c9afb17f3ce7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:57.453683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:57.454627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:40:57.455589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:57.457892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 85/173] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:40:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:57.460402 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4072a8-d74e-41fb-a877-7f6f8d95e236 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:57.461196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4072a8-d74e-41fb-a877-7f6f8d95e236 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:57.461666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4072a8-d74e-41fb-a877-7f6f8d95e236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:57.481317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 89/174] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:56 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:40:57.482189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da6418-409e-46eb-80ba-9d41a08f6d0d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:57.484195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7e4072a8-d74e-41fb-a877-7f6f8d95e236 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:57.485583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7e4072a8-d74e-41fb-a877-7f6f8d95e236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:57.611044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da6418-409e-46eb-80ba-9d41a08f6d0d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:57.620248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:57.631325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 86/175] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:57.631325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4072a8-d74e-41fb-a877-7f6f8d95e236 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:57.667727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed65259-6715-41e3-9db3-7b8737f8cff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:57.668970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed65259-6715-41e3-9db3-7b8737f8cff9 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:57.669475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed65259-6715-41e3-9db3-7b8737f8cff9 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:57.669948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed65259-6715-41e3-9db3-7b8737f8cff9 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:57.670823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed65259-6715-41e3-9db3-7b8737f8cff9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:57.671382 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed65259-6715-41e3-9db3-7b8737f8cff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:57.673212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed65259-6715-41e3-9db3-7b8737f8cff9 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:57.687734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 90/176] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:40:57 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:57.694058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed65259-6715-41e3-9db3-7b8737f8cff9 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:57.764056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:57.815465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4072a8-d74e-41fb-a877-7f6f8d95e236 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:57.848821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed65259-6715-41e3-9db3-7b8737f8cff9 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:57.877612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ba21e-4ef6-41e7-b5c2-6f8499a6c7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:57.878230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ba21e-4ef6-41e7-b5c2-6f8499a6c7be None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:57.878512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ba21e-4ef6-41e7-b5c2-6f8499a6c7be None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:57.878785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ba21e-4ef6-41e7-b5c2-6f8499a6c7be None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:57.880693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ba21e-4ef6-41e7-b5c2-6f8499a6c7be None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:57.882947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ba21e-4ef6-41e7-b5c2-6f8499a6c7be None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:57.885649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 87/177] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:57 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:57.886800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873ca3c-7b3e-4b61-a3c5-7737e75169b8 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:57.887104 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873ca3c-7b3e-4b61-a3c5-7737e75169b8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:57.887380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873ca3c-7b3e-4b61-a3c5-7737e75169b8 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:57.890333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 88/178] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:57 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:57.891444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873ca3c-7b3e-4b61-a3c5-7737e75169b8 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:57.911954 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54894c2f-a81b-4c7a-94fc-a14e84014005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:57.913833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894c2f-a81b-4c7a-94fc-a14e84014005 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:57.914341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894c2f-a81b-4c7a-94fc-a14e84014005 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:57.914833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54894c2f-a81b-4c7a-94fc-a14e84014005 None admin] PATH_INFO: `/v3/users/5109071b2f5d4a8fbd1639a297f22e81` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:57.916297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54894c2f-a81b-4c7a-94fc-a14e84014005 None admin] RBAC: Authorizing `identity:get_user(user_id=5109071b2f5d4a8fbd1639a297f22e81)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:57.918157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54894c2f-a81b-4c7a-94fc-a14e84014005 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:57.920386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 91/179] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:57 2022] GET /identity/v3/users/5109071b2f5d4a8fbd1639a297f22e81 => generated 276 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:40:57.924491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1882d-2541-4d7a-8f33-a34423d215d4 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:57.925340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1882d-2541-4d7a-8f33-a34423d215d4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:57.925815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1882d-2541-4d7a-8f33-a34423d215d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:57.947834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-37d1882d-2541-4d7a-8f33-a34423d215d4 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:57.949344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-37d1882d-2541-4d7a-8f33-a34423d215d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:57.982302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:58.035393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 92/180] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:58.036734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1882d-2541-4d7a-8f33-a34423d215d4 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:58.054616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873ca3c-7b3e-4b61-a3c5-7737e75169b8 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:58.078508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:58.090579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f6fc9-050b-47f8-aa42-07f2881a9be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:58.091235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6fc9-050b-47f8-aa42-07f2881a9be9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.091513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6fc9-050b-47f8-aa42-07f2881a9be9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.091785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6fc9-050b-47f8-aa42-07f2881a9be9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.092513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f6fc9-050b-47f8-aa42-07f2881a9be9 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=a1f41b226cc94cccbb39608d62cb6737, user.id=f6a4ad480e8041b39ccfa602ac02988c) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:58.092819 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f6fc9-050b-47f8-aa42-07f2881a9be9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:58.094246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f6fc9-050b-47f8-aa42-07f2881a9be9 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:58.100473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 89/181] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:40:57 2022] GET /identity/v3/role_assignments?user.id=f6a4ad480e8041b39ccfa602ac02988c&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 610 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:58.101660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfd842-0ec7-47b7-8351-6f582a5f36ab None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.101942 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfd842-0ec7-47b7-8351-6f582a5f36ab None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.102281 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfd842-0ec7-47b7-8351-6f582a5f36ab None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.103514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 90/182] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:58 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:58.109263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba20e6a-0bfe-4019-83fa-7b1067d9045c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.109576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba20e6a-0bfe-4019-83fa-7b1067d9045c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.109814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba20e6a-0bfe-4019-83fa-7b1067d9045c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.129259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-3ba20e6a-0bfe-4019-83fa-7b1067d9045c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:58.131824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3ba20e6a-0bfe-4019-83fa-7b1067d9045c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:58.210673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 91/183] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:40:58.215490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba20e6a-0bfe-4019-83fa-7b1067d9045c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:58.248119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1882d-2541-4d7a-8f33-a34423d215d4 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:58.327620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791666a-2065-40ec-aa2e-079b21ad7686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:58.328730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791666a-2065-40ec-aa2e-079b21ad7686 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.329827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791666a-2065-40ec-aa2e-079b21ad7686 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.330354 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791666a-2065-40ec-aa2e-079b21ad7686 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.368634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:58.377260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba20e6a-0bfe-4019-83fa-7b1067d9045c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:58.395263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:58.406404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791666a-2065-40ec-aa2e-079b21ad7686 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:58.408125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5791666a-2065-40ec-aa2e-079b21ad7686 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:58.415622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8dd6a-91cf-4aaa-bfca-15a61a493843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:58.416462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8dd6a-91cf-4aaa-bfca-15a61a493843 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.417006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8dd6a-91cf-4aaa-bfca-15a61a493843 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.417431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8dd6a-91cf-4aaa-bfca-15a61a493843 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.428664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8dd6a-91cf-4aaa-bfca-15a61a493843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-bfb8dd6a-91cf-4aaa-bfca-15a61a493843 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:58.433757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:58.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:40:58.435683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:40:58.437550 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 92/184] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:40:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:58.439592 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090661e-6ce0-4b6b-afec-87b6102a70fa None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.440025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090661e-6ce0-4b6b-afec-87b6102a70fa None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.440435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e090661e-6ce0-4b6b-afec-87b6102a70fa None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.441849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 93/185] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:40:58 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:40:58.446487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090661e-6ce0-4b6b-afec-87b6102a70fa None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:58.539063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 93/186] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:40:58.540225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791666a-2065-40ec-aa2e-079b21ad7686 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:58.560090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e090661e-6ce0-4b6b-afec-87b6102a70fa None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:58.595176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d502d4-7edc-486d-90f7-165507462bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:58.595735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d502d4-7edc-486d-90f7-165507462bba None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.596016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d502d4-7edc-486d-90f7-165507462bba None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.596285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d502d4-7edc-486d-90f7-165507462bba None admin] PATH_INFO: `/v3/domains/default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.597386 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d502d4-7edc-486d-90f7-165507462bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:58.598820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d502d4-7edc-486d-90f7-165507462bba None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:58.601088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 94/187] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:40:58 2022] GET /identity/v3/domains/default => generated 200 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:58.602192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9ccd8-112d-4fcb-864a-1f3f24be05bf None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.602472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9ccd8-112d-4fcb-864a-1f3f24be05bf None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.602743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9ccd8-112d-4fcb-864a-1f3f24be05bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.604928 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:58.620488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e8f9ccd8-112d-4fcb-864a-1f3f24be05bf None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:40:58.621739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e8f9ccd8-112d-4fcb-864a-1f3f24be05bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:40:58.640353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791666a-2065-40ec-aa2e-079b21ad7686 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:58.662833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:58.673623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952fe74-0105-4079-bb1a-7a245a246a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:58.674230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952fe74-0105-4079-bb1a-7a245a246a3c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.676997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952fe74-0105-4079-bb1a-7a245a246a3c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.676997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952fe74-0105-4079-bb1a-7a245a246a3c None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.676997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9952fe74-0105-4079-bb1a-7a245a246a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:58.676997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9952fe74-0105-4079-bb1a-7a245a246a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:58.677489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9952fe74-0105-4079-bb1a-7a245a246a3c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:58.685219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 94/188] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:58 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:58.687892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952fe74-0105-4079-bb1a-7a245a246a3c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:58.688771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 95/189] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:40:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:40:58.689591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9ccd8-112d-4fcb-864a-1f3f24be05bf None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:58.790811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f9ccd8-112d-4fcb-864a-1f3f24be05bf None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:58.795880 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952fe74-0105-4079-bb1a-7a245a246a3c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:58.825819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:58.826763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11ef12-fa79-4463-aab5-9f32a1ccc259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:58.827373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11ef12-fa79-4463-aab5-9f32a1ccc259 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.827581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11ef12-fa79-4463-aab5-9f32a1ccc259 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.827848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11ef12-fa79-4463-aab5-9f32a1ccc259 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.828963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11ef12-fa79-4463-aab5-9f32a1ccc259 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:58.830397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11ef12-fa79-4463-aab5-9f32a1ccc259 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:58.832114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 96/190] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:58 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:58.832855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11ef12-fa79-4463-aab5-9f32a1ccc259 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:58.839754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168b6fc-83ff-4445-b044-171152b308cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:58.840398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168b6fc-83ff-4445-b044-171152b308cd None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.840678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168b6fc-83ff-4445-b044-171152b308cd None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.840950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168b6fc-83ff-4445-b044-171152b308cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.851870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168b6fc-83ff-4445-b044-171152b308cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-1168b6fc-83ff-4445-b044-171152b308cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:58.856964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:58.857856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:58.858647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 95/191] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:40:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:58.859335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168b6fc-83ff-4445-b044-171152b308cd None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:58.885411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:58.953498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11ef12-fa79-4463-aab5-9f32a1ccc259 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:58.969457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168b6fc-83ff-4445-b044-171152b308cd None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:58.987926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af800d89-da86-4430-9dff-c550e582b8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:58.988541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af800d89-da86-4430-9dff-c550e582b8d0 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:58.988821 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af800d89-da86-4430-9dff-c550e582b8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:58.989089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af800d89-da86-4430-9dff-c550e582b8d0 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:58.989725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af800d89-da86-4430-9dff-c550e582b8d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:58.991215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af800d89-da86-4430-9dff-c550e582b8d0 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:59.001572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af8173-895a-4a70-ae51-7923b5b5d9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:59.002180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af8173-895a-4a70-ae51-7923b5b5d9dc None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:59.002456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af8173-895a-4a70-ae51-7923b5b5d9dc None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:59.002725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af8173-895a-4a70-ae51-7923b5b5d9dc None admin] PATH_INFO: `/v3/users/da8201ad76194976a6ee4942a0e057ca` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:59.017090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af8173-895a-4a70-ae51-7923b5b5d9dc None admin] RBAC: Authorizing `identity:get_user(user_id=da8201ad76194976a6ee4942a0e057ca)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:59.018547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af8173-895a-4a70-ae51-7923b5b5d9dc None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:59.020195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 96/192] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:58 2022] GET /identity/v3/users/da8201ad76194976a6ee4942a0e057ca => generated 284 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:59.020957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af8173-895a-4a70-ae51-7923b5b5d9dc None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:59.065886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:59.078020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 97/193] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:40:58 2022] POST /identity/v3/users => generated 298 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:40:59.078842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af800d89-da86-4430-9dff-c550e582b8d0 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:59.124987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af8173-895a-4a70-ae51-7923b5b5d9dc None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:59.165853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff46cc-52d0-4733-a168-791bff68c8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:59.166475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff46cc-52d0-4733-a168-791bff68c8f3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:59.166750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff46cc-52d0-4733-a168-791bff68c8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:59.167049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff46cc-52d0-4733-a168-791bff68c8f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:59.167782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff46cc-52d0-4733-a168-791bff68c8f3 None admin] RBAC: Adding query filter params (role.id=b7f17471d2c948efa07d02424da68bea, scope.project.id=a1f41b226cc94cccbb39608d62cb6737, user.id=5109071b2f5d4a8fbd1639a297f22e81) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:59.168087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff46cc-52d0-4733-a168-791bff68c8f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:59.172335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff46cc-52d0-4733-a168-791bff68c8f3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:59.177438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 97/194] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:40:59 2022] GET /identity/v3/role_assignments?user.id=5109071b2f5d4a8fbd1639a297f22e81&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=b7f17471d2c948efa07d02424da68bea => generated 610 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:59.184255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff46cc-52d0-4733-a168-791bff68c8f3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:59.199681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:59.199681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af800d89-da86-4430-9dff-c550e582b8d0 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:59.245473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e898273-39c6-4cda-a1dd-d027d2f2a67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:59.246107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e898273-39c6-4cda-a1dd-d027d2f2a67f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:59.246365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e898273-39c6-4cda-a1dd-d027d2f2a67f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:59.246642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e898273-39c6-4cda-a1dd-d027d2f2a67f None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:59.248441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e898273-39c6-4cda-a1dd-d027d2f2a67f None admin] RBAC: Adding query filter params (name=reader) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:59.248441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e898273-39c6-4cda-a1dd-d027d2f2a67f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:59.249288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e898273-39c6-4cda-a1dd-d027d2f2a67f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:59.259204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 98/195] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:40:59 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:59.261099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e898273-39c6-4cda-a1dd-d027d2f2a67f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:59.327595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff46cc-52d0-4733-a168-791bff68c8f3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:59.344575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:59.380873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce6c85-2be8-4be2-91a3-0f5f745af430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:59.381438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce6c85-2be8-4be2-91a3-0f5f745af430 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:59.381714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce6c85-2be8-4be2-91a3-0f5f745af430 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:59.382569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce6c85-2be8-4be2-91a3-0f5f745af430 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:59.395978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce6c85-2be8-4be2-91a3-0f5f745af430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-53ce6c85-2be8-4be2-91a3-0f5f745af430 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:40:59.402195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:40:59.403334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:40:59.404306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 98/196] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:40:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:40:59.405204 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce6c85-2be8-4be2-91a3-0f5f745af430 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:59.414353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e898273-39c6-4cda-a1dd-d027d2f2a67f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:59.461223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00ab83-79ae-4cd0-818e-daf752faff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:59.461841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00ab83-79ae-4cd0-818e-daf752faff40 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:59.462681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00ab83-79ae-4cd0-818e-daf752faff40 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:59.462952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00ab83-79ae-4cd0-818e-daf752faff40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:59.500556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:59.537715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce6c85-2be8-4be2-91a3-0f5f745af430 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:59.559908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00ab83-79ae-4cd0-818e-daf752faff40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:59.562259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00ab83-79ae-4cd0-818e-daf752faff40 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:59.586988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542a93b-f936-4375-b865-87581b545fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:59.588047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542a93b-f936-4375-b865-87581b545fcd None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:59.588837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542a93b-f936-4375-b865-87581b545fcd None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:59.589113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542a93b-f936-4375-b865-87581b545fcd None admin] PATH_INFO: `/v3/users/ef3335126e8640d08c45ab4b702f3891` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:59.590308 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5542a93b-f936-4375-b865-87581b545fcd None admin] RBAC: Authorizing `identity:get_user(user_id=ef3335126e8640d08c45ab4b702f3891)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:59.592243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5542a93b-f936-4375-b865-87581b545fcd None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:59.598644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 99/197] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:59 2022] GET /identity/v3/users/ef3335126e8640d08c45ab4b702f3891 => generated 290 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:59.600384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542a93b-f936-4375-b865-87581b545fcd None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:59.654597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:59.705598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 99/198] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:59 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:40:59.705598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00ab83-79ae-4cd0-818e-daf752faff40 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:59.735506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542a93b-f936-4375-b865-87581b545fcd None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:59.780163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea93e9-e259-4176-8480-2c771849aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:59.781299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea93e9-e259-4176-8480-2c771849aeaa None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:59.781763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:59.782587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea93e9-e259-4176-8480-2c771849aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:59.783190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea93e9-e259-4176-8480-2c771849aeaa None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:59.784063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea93e9-e259-4176-8480-2c771849aeaa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:40:59.784562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea93e9-e259-4176-8480-2c771849aeaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:59.786109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edea93e9-e259-4176-8480-2c771849aeaa None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:59.795437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 100/199] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:40:59 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:59.796722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea93e9-e259-4176-8480-2c771849aeaa None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:59.825660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00ab83-79ae-4cd0-818e-daf752faff40 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:59.861206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db921226-40b2-4fc2-8a2b-4056cda96f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:59.861832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db921226-40b2-4fc2-8a2b-4056cda96f21 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:59.862140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db921226-40b2-4fc2-8a2b-4056cda96f21 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:59.862416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db921226-40b2-4fc2-8a2b-4056cda96f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:59.906318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:40:59.910483 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea93e9-e259-4176-8480-2c771849aeaa None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:40:59.933278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db921226-40b2-4fc2-8a2b-4056cda96f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:59.934873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db921226-40b2-4fc2-8a2b-4056cda96f21 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:59.948735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd30712-f0df-4375-bce9-5eff28aeae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:40:59.949761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd30712-f0df-4375-bce9-5eff28aeae6b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:40:59.950258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd30712-f0df-4375-bce9-5eff28aeae6b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:40:59.950731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd30712-f0df-4375-bce9-5eff28aeae6b None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:40:59.953111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd30712-f0df-4375-bce9-5eff28aeae6b None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:40:59.954809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd30712-f0df-4375-bce9-5eff28aeae6b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:40:59.956784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 101/200] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:40:59 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:40:59.957761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd30712-f0df-4375-bce9-5eff28aeae6b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:40:59.999896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:00.048065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 100/201] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:40:59 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:00.048962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db921226-40b2-4fc2-8a2b-4056cda96f21 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:00.068023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd30712-f0df-4375-bce9-5eff28aeae6b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:00.105750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad05a2-d937-45d8-af6e-d4bd9c67f3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:00.106797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad05a2-d937-45d8-af6e-d4bd9c67f3b5 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:00.107308 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad05a2-d937-45d8-af6e-d4bd9c67f3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:00.107774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ad05a2-d937-45d8-af6e-d4bd9c67f3b5 None admin] PATH_INFO: `/v3/users/da8201ad76194976a6ee4942a0e057ca` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:00.109338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad05a2-d937-45d8-af6e-d4bd9c67f3b5 None admin] RBAC: Authorizing `identity:get_user(user_id=da8201ad76194976a6ee4942a0e057ca)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:00.111049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ad05a2-d937-45d8-af6e-d4bd9c67f3b5 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:00.113004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 102/202] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:40:59 2022] GET /identity/v3/users/da8201ad76194976a6ee4942a0e057ca => generated 284 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:00.113939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad05a2-d937-45d8-af6e-d4bd9c67f3b5 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:00.151336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db921226-40b2-4fc2-8a2b-4056cda96f21 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:00.152670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:00.191231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77671f-c6a1-4d46-b8ec-ab86ad4e9569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:00.191985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77671f-c6a1-4d46-b8ec-ab86ad4e9569 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:00.192430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77671f-c6a1-4d46-b8ec-ab86ad4e9569 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:00.192747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77671f-c6a1-4d46-b8ec-ab86ad4e9569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:00.193631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77671f-c6a1-4d46-b8ec-ab86ad4e9569 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b, user.id=da8201ad76194976a6ee4942a0e057ca) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:00.194137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77671f-c6a1-4d46-b8ec-ab86ad4e9569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:00.195838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77671f-c6a1-4d46-b8ec-ab86ad4e9569 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:00.201708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 101/203] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:00 2022] GET /identity/v3/role_assignments?user.id=da8201ad76194976a6ee4942a0e057ca&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:00.208357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77671f-c6a1-4d46-b8ec-ab86ad4e9569 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:00.236639 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ad05a2-d937-45d8-af6e-d4bd9c67f3b5 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:00.268187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:00.278848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917d12d-20c0-4fd5-9798-070b6fccae2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:00.279855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917d12d-20c0-4fd5-9798-070b6fccae2b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:00.284459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917d12d-20c0-4fd5-9798-070b6fccae2b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:00.284459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917d12d-20c0-4fd5-9798-070b6fccae2b None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:00.284459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917d12d-20c0-4fd5-9798-070b6fccae2b None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:00.284459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917d12d-20c0-4fd5-9798-070b6fccae2b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:00.285607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 103/204] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:00 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:00.296602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917d12d-20c0-4fd5-9798-070b6fccae2b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:00.403365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77671f-c6a1-4d46-b8ec-ab86ad4e9569 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:00.418166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917d12d-20c0-4fd5-9798-070b6fccae2b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:00.421842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:00.459913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0bd42-2585-4700-ab68-de91d93ddff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:00.461407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bd42-2585-4700-ab68-de91d93ddff0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:00.461899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bd42-2585-4700-ab68-de91d93ddff0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:00.462535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0bd42-2585-4700-ab68-de91d93ddff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:00.463615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0bd42-2585-4700-ab68-de91d93ddff0 None admin] RBAC: Adding query filter params (role.id=0f01d78323f24797b5b1f2c555ea4c4c, scope.project.id=a1f41b226cc94cccbb39608d62cb6737, user.id=ef3335126e8640d08c45ab4b702f3891) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:00.464176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0bd42-2585-4700-ab68-de91d93ddff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:00.465872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0bd42-2585-4700-ab68-de91d93ddff0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:00.473599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 104/205] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:00 2022] GET /identity/v3/role_assignments?user.id=ef3335126e8640d08c45ab4b702f3891&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=0f01d78323f24797b5b1f2c555ea4c4c => generated 610 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:00.484553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bccb37-1c63-491d-ab64-925a61623e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:00.486496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bccb37-1c63-491d-ab64-925a61623e7e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:00.486496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bccb37-1c63-491d-ab64-925a61623e7e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:00.486496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bccb37-1c63-491d-ab64-925a61623e7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:00.567857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bccb37-1c63-491d-ab64-925a61623e7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:00.569748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bccb37-1c63-491d-ab64-925a61623e7e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:00.601884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:00.693036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 102/206] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:00 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:00.699548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0bd42-2585-4700-ab68-de91d93ddff0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:00.818781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0bd42-2585-4700-ab68-de91d93ddff0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:00.855575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af825d-4d8d-4364-a82f-1bc376898bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:00.856558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af825d-4d8d-4364-a82f-1bc376898bce None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:00.857028 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af825d-4d8d-4364-a82f-1bc376898bce None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:00.857486 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af825d-4d8d-4364-a82f-1bc376898bce None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:00.858826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af825d-4d8d-4364-a82f-1bc376898bce None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:00.860552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af825d-4d8d-4364-a82f-1bc376898bce None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:00.862727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 105/207] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:00 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:00.869267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bccb37-1c63-491d-ab64-925a61623e7e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:00.979357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bccb37-1c63-491d-ab64-925a61623e7e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:01.004996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:01.022634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e7df3f-07b4-4d4e-acb6-476e608b97ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:01.027451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7df3f-07b4-4d4e-acb6-476e608b97ce None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:01.027451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7df3f-07b4-4d4e-acb6-476e608b97ce None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:01.027451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e7df3f-07b4-4d4e-acb6-476e608b97ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:01.027451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7df3f-07b4-4d4e-acb6-476e608b97ce None admin] RBAC: Adding query filter params (role.id=b7f17471d2c948efa07d02424da68bea, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b, user.id=da8201ad76194976a6ee4942a0e057ca) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:01.027451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7df3f-07b4-4d4e-acb6-476e608b97ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:01.027451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e7df3f-07b4-4d4e-acb6-476e608b97ce None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:01.038685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 103/208] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:00 2022] GET /identity/v3/role_assignments?user.id=da8201ad76194976a6ee4942a0e057ca&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=b7f17471d2c948efa07d02424da68bea => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:01.366572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeb134-24c4-45be-98b5-62fe14cf52fc None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:01.366953 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeb134-24c4-45be-98b5-62fe14cf52fc None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:01.367160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eeb134-24c4-45be-98b5-62fe14cf52fc None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:01.369060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 106/209] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:01.374369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f384c-6552-46cc-a4fa-4b3183fd4306 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:01.374807 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f384c-6552-46cc-a4fa-4b3183fd4306 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:01.374895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f384c-6552-46cc-a4fa-4b3183fd4306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:01.393171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-978f384c-6552-46cc-a4fa-4b3183fd4306 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:01.394367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-978f384c-6552-46cc-a4fa-4b3183fd4306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:01.407378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2c456-8e01-4828-b35c-8f8eb0293154 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:01.407671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2c456-8e01-4828-b35c-8f8eb0293154 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:01.407951 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2c456-8e01-4828-b35c-8f8eb0293154 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:01.410089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 107/210] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:01.416183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff1fe0-0ba8-4e75-bfb5-ffcfb9d9b2fc None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:01.416464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff1fe0-0ba8-4e75-bfb5-ffcfb9d9b2fc None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:01.416738 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff1fe0-0ba8-4e75-bfb5-ffcfb9d9b2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:01.437958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c2ff1fe0-0ba8-4e75-bfb5-ffcfb9d9b2fc None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:01.439595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c2ff1fe0-0ba8-4e75-bfb5-ffcfb9d9b2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:01.464264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 104/211] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:01.486784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:01.519792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 108/212] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:01.753449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f384c-6552-46cc-a4fa-4b3183fd4306 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:01.809832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff1fe0-0ba8-4e75-bfb5-ffcfb9d9b2fc None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:01.868061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f384c-6552-46cc-a4fa-4b3183fd4306 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:01.891205 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:01.909436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f37f0-8b77-4ce8-af43-a1851656dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:01.910077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f37f0-8b77-4ce8-af43-a1851656dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:01.910356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f37f0-8b77-4ce8-af43-a1851656dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:01.910621 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f37f0-8b77-4ce8-af43-a1851656dcb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:01.911793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f37f0-8b77-4ce8-af43-a1851656dcb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:01.913245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f37f0-8b77-4ce8-af43-a1851656dcb4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:01.914838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 105/213] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:01 2022] GET /identity/v3/domains/default => generated 200 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:01.916886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff1fe0-0ba8-4e75-bfb5-ffcfb9d9b2fc None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:01.926478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f37f0-8b77-4ce8-af43-a1851656dcb4 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:01.956110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100e953-24a2-4766-9e61-d5b38db9844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:01.956744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100e953-24a2-4766-9e61-d5b38db9844a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:01.957023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100e953-24a2-4766-9e61-d5b38db9844a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:01.957296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100e953-24a2-4766-9e61-d5b38db9844a None admin] PATH_INFO: `/v3/roles/member` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:01.968560 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100e953-24a2-4766-9e61-d5b38db9844a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-a100e953-24a2-4766-9e61-d5b38db9844a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:01.973298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:01.974364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:01.975214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 109/214] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:01.980684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100e953-24a2-4766-9e61-d5b38db9844a None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:02.011890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:02.057877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f37f0-8b77-4ce8-af43-a1851656dcb4 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:02.102187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516051bc-bfb7-4b4d-a55b-9e95f7cbd836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:02.103799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516051bc-bfb7-4b4d-a55b-9e95f7cbd836 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:02.104463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516051bc-bfb7-4b4d-a55b-9e95f7cbd836 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:02.104944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516051bc-bfb7-4b4d-a55b-9e95f7cbd836 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:02.105919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516051bc-bfb7-4b4d-a55b-9e95f7cbd836 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:02.108128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516051bc-bfb7-4b4d-a55b-9e95f7cbd836 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:02.115326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100e953-24a2-4766-9e61-d5b38db9844a None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:02.156699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034a595-fc1d-44b0-a24d-abf99ddb2623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:02.157419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034a595-fc1d-44b0-a24d-abf99ddb2623 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:02.157790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034a595-fc1d-44b0-a24d-abf99ddb2623 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:02.158134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a034a595-fc1d-44b0-a24d-abf99ddb2623 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:02.158867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034a595-fc1d-44b0-a24d-abf99ddb2623 None admin] RBAC: Adding query filter params (name=member) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:02.159261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034a595-fc1d-44b0-a24d-abf99ddb2623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:02.160752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a034a595-fc1d-44b0-a24d-abf99ddb2623 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:02.169825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 110/215] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:01 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:02.175739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034a595-fc1d-44b0-a24d-abf99ddb2623 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:02.207162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 106/216] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:41:01 2022] POST /identity/v3/users => generated 298 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:41:02.284962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:02.299121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a034a595-fc1d-44b0-a24d-abf99ddb2623 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:02.340978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356c797-3e63-4aa0-9d46-fc176a1924e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:02.341897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356c797-3e63-4aa0-9d46-fc176a1924e9 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:02.342279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356c797-3e63-4aa0-9d46-fc176a1924e9 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:02.342652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356c797-3e63-4aa0-9d46-fc176a1924e9 None admin] PATH_INFO: `/v3/users/6b67c397b28b4ee4b1f4c3bd58f2c239` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:02.358592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356c797-3e63-4aa0-9d46-fc176a1924e9 None admin] RBAC: Authorizing `identity:get_user(user_id=6b67c397b28b4ee4b1f4c3bd58f2c239)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:02.360264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356c797-3e63-4aa0-9d46-fc176a1924e9 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:02.362381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 111/217] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:02 2022] GET /identity/v3/users/6b67c397b28b4ee4b1f4c3bd58f2c239 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:02.368076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516051bc-bfb7-4b4d-a55b-9e95f7cbd836 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:02.419212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:02.482010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516051bc-bfb7-4b4d-a55b-9e95f7cbd836 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:02.491104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e48cfd-d73c-47fa-b19d-97f276cfbc1f None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:02.491579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e48cfd-d73c-47fa-b19d-97f276cfbc1f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:02.491653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e48cfd-d73c-47fa-b19d-97f276cfbc1f None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:02.492994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 112/218] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:02.498587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557c296-4161-4cce-a9ea-6e30180bc988 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:02.498865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557c296-4161-4cce-a9ea-6e30180bc988 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:02.499255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557c296-4161-4cce-a9ea-6e30180bc988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:02.521107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c557c296-4161-4cce-a9ea-6e30180bc988 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:02.522646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c557c296-4161-4cce-a9ea-6e30180bc988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:02.533203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca2db-6aaa-4e4e-860f-3bfbd10c1491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:02.533824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca2db-6aaa-4e4e-860f-3bfbd10c1491 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:02.534108 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca2db-6aaa-4e4e-860f-3bfbd10c1491 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:02.534382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca2db-6aaa-4e4e-860f-3bfbd10c1491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:02.619073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 113/219] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:02.628549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ca2db-6aaa-4e4e-860f-3bfbd10c1491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:02.630015 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ca2db-6aaa-4e4e-860f-3bfbd10c1491 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:02.676119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:02.745335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 107/220] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:02.746871 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557c296-4161-4cce-a9ea-6e30180bc988 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:02.841261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:02.866621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557c296-4161-4cce-a9ea-6e30180bc988 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:02.914188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff8b6a-d613-4405-b6d6-e76ae227074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:02.914756 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff8b6a-d613-4405-b6d6-e76ae227074f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:02.915053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff8b6a-d613-4405-b6d6-e76ae227074f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:02.915332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff8b6a-d613-4405-b6d6-e76ae227074f None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:02.917331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff8b6a-d613-4405-b6d6-e76ae227074f None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:02.918794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff8b6a-d613-4405-b6d6-e76ae227074f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:02.920648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 114/221] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:02 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:02.926753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca2db-6aaa-4e4e-860f-3bfbd10c1491 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:02.936558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff8b6a-d613-4405-b6d6-e76ae227074f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:03.094057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff8b6a-d613-4405-b6d6-e76ae227074f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:03.112880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca2db-6aaa-4e4e-860f-3bfbd10c1491 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:03.138234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd1f5f-0bcc-4069-8f35-4f4df3facba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:03.138940 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd1f5f-0bcc-4069-8f35-4f4df3facba4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:03.139252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd1f5f-0bcc-4069-8f35-4f4df3facba4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:03.139528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd1f5f-0bcc-4069-8f35-4f4df3facba4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:03.151292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd1f5f-0bcc-4069-8f35-4f4df3facba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-44fd1f5f-0bcc-4069-8f35-4f4df3facba4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:03.156466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:03.157371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:03.158194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 115/222] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:41:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:03.159382 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3ddaa-0f00-44c1-bd44-5b23ed75209e None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:03.159637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3ddaa-0f00-44c1-bd44-5b23ed75209e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:03.159957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3ddaa-0f00-44c1-bd44-5b23ed75209e None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:03.161337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 116/223] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:03.162979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbf9bb-1ff6-4d82-8890-b28d9e0f9c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:03.163887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf9bb-1ff6-4d82-8890-b28d9e0f9c8e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:03.164459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf9bb-1ff6-4d82-8890-b28d9e0f9c8e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:03.164926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf9bb-1ff6-4d82-8890-b28d9e0f9c8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:03.166825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbf9bb-1ff6-4d82-8890-b28d9e0f9c8e None admin] RBAC: Adding query filter params (role.id=df27fa65390e4de78fa117773dc73555, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b, user.id=6b67c397b28b4ee4b1f4c3bd58f2c239) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:03.167373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbf9bb-1ff6-4d82-8890-b28d9e0f9c8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:03.167498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3ddaa-0f00-44c1-bd44-5b23ed75209e None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:03.169129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbf9bb-1ff6-4d82-8890-b28d9e0f9c8e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:03.172742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:03.182148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 108/224] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:02 2022] GET /identity/v3/role_assignments?user.id=6b67c397b28b4ee4b1f4c3bd58f2c239&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=df27fa65390e4de78fa117773dc73555 => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:03.201727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8252e-66a0-4ed7-96dc-cc013c1e1c70 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:03.201727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8252e-66a0-4ed7-96dc-cc013c1e1c70 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:03.201727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8252e-66a0-4ed7-96dc-cc013c1e1c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:03.210699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:03.223951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-4ed8252e-66a0-4ed7-96dc-cc013c1e1c70 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:03.225195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4ed8252e-66a0-4ed7-96dc-cc013c1e1c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:03.349231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 109/225] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:03.383328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3ddaa-0f00-44c1-bd44-5b23ed75209e None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:03.421487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da220f1a-255b-4b4b-afa8-0b4002b5f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:03.422021 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da220f1a-255b-4b4b-afa8-0b4002b5f631 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:03.422301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da220f1a-255b-4b4b-afa8-0b4002b5f631 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:03.422825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da220f1a-255b-4b4b-afa8-0b4002b5f631 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:03.423578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da220f1a-255b-4b4b-afa8-0b4002b5f631 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:03.423879 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da220f1a-255b-4b4b-afa8-0b4002b5f631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:03.425359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da220f1a-255b-4b4b-afa8-0b4002b5f631 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:03.434962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 117/226] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:03 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:03.440195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8252e-66a0-4ed7-96dc-cc013c1e1c70 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:03.548268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8252e-66a0-4ed7-96dc-cc013c1e1c70 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:03.584462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e8820-3843-4d04-916b-a8e2698f2b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:03.585168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e8820-3843-4d04-916b-a8e2698f2b12 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:03.585450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e8820-3843-4d04-916b-a8e2698f2b12 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:03.585728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e8820-3843-4d04-916b-a8e2698f2b12 None admin] PATH_INFO: `/v3/users/da8201ad76194976a6ee4942a0e057ca` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:03.586974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e8820-3843-4d04-916b-a8e2698f2b12 None admin] RBAC: Authorizing `identity:get_user(user_id=da8201ad76194976a6ee4942a0e057ca)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:03.588501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e8820-3843-4d04-916b-a8e2698f2b12 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:03.590401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 110/227] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:03 2022] GET /identity/v3/users/da8201ad76194976a6ee4942a0e057ca => generated 284 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:03.595597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da220f1a-255b-4b4b-afa8-0b4002b5f631 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:03.669577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:03.706676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da220f1a-255b-4b4b-afa8-0b4002b5f631 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:03.724862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e8820-3843-4d04-916b-a8e2698f2b12 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:03.729527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:03.746552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf9d73-1206-42bc-a64b-4f1e69477b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:03.747231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf9d73-1206-42bc-a64b-4f1e69477b37 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:03.747517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf9d73-1206-42bc-a64b-4f1e69477b37 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:03.748314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf9d73-1206-42bc-a64b-4f1e69477b37 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:03.749774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf9d73-1206-42bc-a64b-4f1e69477b37 None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:03.751947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf9d73-1206-42bc-a64b-4f1e69477b37 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:03.755929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 118/228] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:03 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:03.763978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf9d73-1206-42bc-a64b-4f1e69477b37 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:03.855437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e8820-3843-4d04-916b-a8e2698f2b12 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:03.896408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507fcd3-a1c6-4b98-bee2-d60041558dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:03.897665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507fcd3-a1c6-4b98-bee2-d60041558dd6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:03.898148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507fcd3-a1c6-4b98-bee2-d60041558dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:03.898643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507fcd3-a1c6-4b98-bee2-d60041558dd6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:03.911531 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507fcd3-a1c6-4b98-bee2-d60041558dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-4507fcd3-a1c6-4b98-bee2-d60041558dd6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:03.918792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:03.919795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:03.920616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:03.924220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 111/229] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:41:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:03.931832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507fcd3-a1c6-4b98-bee2-d60041558dd6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:04.000500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf9d73-1206-42bc-a64b-4f1e69477b37 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:04.060315 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:04.080678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddb0a3-c805-4e77-ad45-3515d1e1fb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:04.081684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddb0a3-c805-4e77-ad45-3515d1e1fb96 None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:04.082194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddb0a3-c805-4e77-ad45-3515d1e1fb96 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:04.082674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddb0a3-c805-4e77-ad45-3515d1e1fb96 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/users/da8201ad76194976a6ee4942a0e057ca/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:04.091996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddb0a3-c805-4e77-ad45-3515d1e1fb96 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff12bdfc9e9745779c7f414a2d09d14b, user_id=da8201ad76194976a6ee4942a0e057ca, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:04.093698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddb0a3-c805-4e77-ad45-3515d1e1fb96 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:04.096483 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:04.100279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507fcd3-a1c6-4b98-bee2-d60041558dd6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:04.136240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038bf45-0b05-4e92-b03a-625318d7951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:04.136855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038bf45-0b05-4e92-b03a-625318d7951b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:04.137137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038bf45-0b05-4e92-b03a-625318d7951b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:04.137411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038bf45-0b05-4e92-b03a-625318d7951b None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:04.138073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038bf45-0b05-4e92-b03a-625318d7951b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:04.138379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038bf45-0b05-4e92-b03a-625318d7951b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:04.139777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038bf45-0b05-4e92-b03a-625318d7951b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:04.148450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 112/230] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:03 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:04.154485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038bf45-0b05-4e92-b03a-625318d7951b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:04.195072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 119/231] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:41:03 2022] PUT /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/users/da8201ad76194976a6ee4942a0e057ca/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:04.266900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038bf45-0b05-4e92-b03a-625318d7951b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:04.295891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:04.304882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da95ca2-caaa-4300-bec2-0e1c6001bfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:04.305536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da95ca2-caaa-4300-bec2-0e1c6001bfb4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:04.305814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da95ca2-caaa-4300-bec2-0e1c6001bfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:04.306085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da95ca2-caaa-4300-bec2-0e1c6001bfb4 None admin] PATH_INFO: `/v3/users/da8201ad76194976a6ee4942a0e057ca` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:04.307981 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da95ca2-caaa-4300-bec2-0e1c6001bfb4 None admin] RBAC: Authorizing `identity:get_user(user_id=da8201ad76194976a6ee4942a0e057ca)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:04.309721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da95ca2-caaa-4300-bec2-0e1c6001bfb4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:04.311706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 113/232] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:04 2022] GET /identity/v3/users/da8201ad76194976a6ee4942a0e057ca => generated 284 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:04.317687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddb0a3-c805-4e77-ad45-3515d1e1fb96 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:04.454035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddb0a3-c805-4e77-ad45-3515d1e1fb96 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:04.498311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ef27e-e94a-4428-aaf7-4e7bd223ef99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:04.498959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef27e-e94a-4428-aaf7-4e7bd223ef99 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:04.499331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef27e-e94a-4428-aaf7-4e7bd223ef99 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:04.499607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef27e-e94a-4428-aaf7-4e7bd223ef99 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:04.500831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ef27e-e94a-4428-aaf7-4e7bd223ef99 None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:04.502321 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ef27e-e94a-4428-aaf7-4e7bd223ef99 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:04.504295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 120/233] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:04 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:04.509781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da95ca2-caaa-4300-bec2-0e1c6001bfb4 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:04.676518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da95ca2-caaa-4300-bec2-0e1c6001bfb4 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:04.704264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf79e9f-705a-4d35-90e6-93c3781934d0 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:04.704772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf79e9f-705a-4d35-90e6-93c3781934d0 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:04.704858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf79e9f-705a-4d35-90e6-93c3781934d0 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:04.706125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 121/234] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:41:04.711629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e50d7-fe8c-4da3-b5f4-bb0bb9bd154d None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:04.711913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e50d7-fe8c-4da3-b5f4-bb0bb9bd154d None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:04.712188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e50d7-fe8c-4da3-b5f4-bb0bb9bd154d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:04.729976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06c015e-2fdb-4b88-8e4a-bd41a0eae2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:04.730891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c015e-2fdb-4b88-8e4a-bd41a0eae2a2 None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:04.731368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c015e-2fdb-4b88-8e4a-bd41a0eae2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:04.731780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06c015e-2fdb-4b88-8e4a-bd41a0eae2a2 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/users/da8201ad76194976a6ee4942a0e057ca/roles/b7f17471d2c948efa07d02424da68bea` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:04.733482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-040e50d7-fe8c-4da3-b5f4-bb0bb9bd154d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:04.734377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c015e-2fdb-4b88-8e4a-bd41a0eae2a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff12bdfc9e9745779c7f414a2d09d14b, user_id=da8201ad76194976a6ee4942a0e057ca, role_id=b7f17471d2c948efa07d02424da68bea)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:04.734682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-040e50d7-fe8c-4da3-b5f4-bb0bb9bd154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:04.736161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06c015e-2fdb-4b88-8e4a-bd41a0eae2a2 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:04.801840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:04.813773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 122/235] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:04.837821 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 114/236] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:41:04 2022] PUT /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/users/da8201ad76194976a6ee4942a0e057ca/roles/b7f17471d2c948efa07d02424da68bea => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:04.891875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e477-276d-4929-ad78-4a463055a656 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:04.892177 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e477-276d-4929-ad78-4a463055a656 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:04.892429 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445e477-276d-4929-ad78-4a463055a656 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:04.894205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 123/237] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:04.900745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb088b-5122-4d61-803e-c8ecd49cf724 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:04.901035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb088b-5122-4d61-803e-c8ecd49cf724 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:04.901996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb088b-5122-4d61-803e-c8ecd49cf724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:04.930972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-71cb088b-5122-4d61-803e-c8ecd49cf724 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:04.932120 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-71cb088b-5122-4d61-803e-c8ecd49cf724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:05.002739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:05.023292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 115/238] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:04 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:05.154518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445e477-276d-4929-ad78-4a463055a656 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:05.275596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445e477-276d-4929-ad78-4a463055a656 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:05.313217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de9eda-2441-44bb-8d8d-77813cc78af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:05.314076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de9eda-2441-44bb-8d8d-77813cc78af6 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:05.314498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de9eda-2441-44bb-8d8d-77813cc78af6 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:05.314935 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8de9eda-2441-44bb-8d8d-77813cc78af6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:05.325556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8de9eda-2441-44bb-8d8d-77813cc78af6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b8de9eda-2441-44bb-8d8d-77813cc78af6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:05.330559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:05.331663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:05.332672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:05.334875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 124/239] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:05.341508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb088b-5122-4d61-803e-c8ecd49cf724 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:05.394213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de9eda-2441-44bb-8d8d-77813cc78af6 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:05.469245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb088b-5122-4d61-803e-c8ecd49cf724 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:05.490776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:05.506971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e9542a-9dce-44b8-b276-faf73fcb8f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:05.527221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9542a-9dce-44b8-b276-faf73fcb8f41 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:05.527221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9542a-9dce-44b8-b276-faf73fcb8f41 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:05.527221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e9542a-9dce-44b8-b276-faf73fcb8f41 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:05.527221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9542a-9dce-44b8-b276-faf73fcb8f41 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:05.527221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9542a-9dce-44b8-b276-faf73fcb8f41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:05.527221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e9542a-9dce-44b8-b276-faf73fcb8f41 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:05.528067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 116/240] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:05 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:05.528067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8e559-1686-4474-8d3d-3005d86b47a3 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:05.528067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8e559-1686-4474-8d3d-3005d86b47a3 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:05.528067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a8e559-1686-4474-8d3d-3005d86b47a3 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:05.529267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 117/241] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:05 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:05.530226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8e559-1686-4474-8d3d-3005d86b47a3 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:05.553164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8de9eda-2441-44bb-8d8d-77813cc78af6 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:05.580926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:05.593728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd2178-bb9f-4642-a134-c1486ed4b6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:05.594643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd2178-bb9f-4642-a134-c1486ed4b6b6 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:05.595298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd2178-bb9f-4642-a134-c1486ed4b6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:05.595752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd2178-bb9f-4642-a134-c1486ed4b6b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:05.597074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd2178-bb9f-4642-a134-c1486ed4b6b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:05.598708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd2178-bb9f-4642-a134-c1486ed4b6b6 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:05.600858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 125/242] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:05 2022] GET /identity/v3/domains/default => generated 200 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:05.602409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fe4e1-7152-42ed-bb19-15a79dca54c3 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:05.602869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fe4e1-7152-42ed-bb19-15a79dca54c3 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:05.603446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fe4e1-7152-42ed-bb19-15a79dca54c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:05.624359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-db8fe4e1-7152-42ed-bb19-15a79dca54c3 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:05.625947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-db8fe4e1-7152-42ed-bb19-15a79dca54c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:05.639463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a8e559-1686-4474-8d3d-3005d86b47a3 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:05.675615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21a2e0-0375-41b6-8933-16462b49a29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:05.676691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21a2e0-0375-41b6-8933-16462b49a29e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:05.676691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21a2e0-0375-41b6-8933-16462b49a29e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:05.677054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21a2e0-0375-41b6-8933-16462b49a29e None admin] PATH_INFO: `/v3/users/2ebde5c0c4a84150bc51fb6358ac5e4b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:05.692663 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21a2e0-0375-41b6-8933-16462b49a29e None admin] RBAC: Authorizing `identity:get_user(user_id=2ebde5c0c4a84150bc51fb6358ac5e4b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:05.694335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21a2e0-0375-41b6-8933-16462b49a29e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:05.696391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 118/243] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:05 2022] GET /identity/v3/users/2ebde5c0c4a84150bc51fb6358ac5e4b => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:05.702166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21a2e0-0375-41b6-8933-16462b49a29e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:05.710680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 126/244] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:05.712324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fe4e1-7152-42ed-bb19-15a79dca54c3 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:05.819463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:05.853801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21a2e0-0375-41b6-8933-16462b49a29e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:05.856912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:05.889806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f1638-46f4-4d4a-ada0-20c18f0dc9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:05.890414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f1638-46f4-4d4a-ada0-20c18f0dc9de None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:05.890692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f1638-46f4-4d4a-ada0-20c18f0dc9de None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:05.890959 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f1638-46f4-4d4a-ada0-20c18f0dc9de None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:05.891650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f1638-46f4-4d4a-ada0-20c18f0dc9de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:05.893226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f1638-46f4-4d4a-ada0-20c18f0dc9de None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:05.927852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fe4e1-7152-42ed-bb19-15a79dca54c3 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:05.970395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93d64f-03cd-4cbd-8456-dde468bc2fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:05.971373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93d64f-03cd-4cbd-8456-dde468bc2fe9 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:05.971892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93d64f-03cd-4cbd-8456-dde468bc2fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:05.972365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec93d64f-03cd-4cbd-8456-dde468bc2fe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:05.994233 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 119/245] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:41:05 2022] POST /identity/v3/users => generated 294 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:41:06.034246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f1638-46f4-4d4a-ada0-20c18f0dc9de None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:06.053164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93d64f-03cd-4cbd-8456-dde468bc2fe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:06.055122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec93d64f-03cd-4cbd-8456-dde468bc2fe9 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:06.112990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:06.150640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f1638-46f4-4d4a-ada0-20c18f0dc9de None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:06.172895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 127/246] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:05 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:06.180819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93d64f-03cd-4cbd-8456-dde468bc2fe9 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:06.198697 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37760123-2b1c-4793-992c-5ee7182bc728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:06.200382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37760123-2b1c-4793-992c-5ee7182bc728 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:06.200663 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37760123-2b1c-4793-992c-5ee7182bc728 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:06.200934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37760123-2b1c-4793-992c-5ee7182bc728 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:06.213267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37760123-2b1c-4793-992c-5ee7182bc728 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-37760123-2b1c-4793-992c-5ee7182bc728 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:06.219186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:06.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:06.221373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 120/247] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:06.226242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37760123-2b1c-4793-992c-5ee7182bc728 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:06.251148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:06.340830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec93d64f-03cd-4cbd-8456-dde468bc2fe9 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:06.342233 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37760123-2b1c-4793-992c-5ee7182bc728 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:06.383808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730afca9-ed92-41c5-9c25-46cd5105aa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:06.384300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730afca9-ed92-41c5-9c25-46cd5105aa9c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:06.384577 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730afca9-ed92-41c5-9c25-46cd5105aa9c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:06.384846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730afca9-ed92-41c5-9c25-46cd5105aa9c None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:06.385530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730afca9-ed92-41c5-9c25-46cd5105aa9c None admin] RBAC: Adding query filter params (name=member) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:06.385808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730afca9-ed92-41c5-9c25-46cd5105aa9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:06.388227 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730afca9-ed92-41c5-9c25-46cd5105aa9c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:06.390318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c5586-fcaa-4f84-bb6e-893a6b68ef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:06.391612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c5586-fcaa-4f84-bb6e-893a6b68ef34 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:06.392109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c5586-fcaa-4f84-bb6e-893a6b68ef34 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:06.392581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c5586-fcaa-4f84-bb6e-893a6b68ef34 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:06.394925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c5586-fcaa-4f84-bb6e-893a6b68ef34 None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:06.396636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c5586-fcaa-4f84-bb6e-893a6b68ef34 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:06.398849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 121/249] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:06 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:06.399014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 128/249] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:06 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:06.405530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730afca9-ed92-41c5-9c25-46cd5105aa9c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:06.406778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c5586-fcaa-4f84-bb6e-893a6b68ef34 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:06.428125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:06.497177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:06.517312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c5586-fcaa-4f84-bb6e-893a6b68ef34 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:06.517633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730afca9-ed92-41c5-9c25-46cd5105aa9c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:06.555204 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862581c5-606b-438d-9ffc-e5655322071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:06.555549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8228da9-35d7-417a-a5db-c0b8af49e4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:06.556084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862581c5-606b-438d-9ffc-e5655322071f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:06.556518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862581c5-606b-438d-9ffc-e5655322071f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:06.556851 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862581c5-606b-438d-9ffc-e5655322071f None admin] PATH_INFO: `/v3/users/6b67c397b28b4ee4b1f4c3bd58f2c239` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:06.557241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228da9-35d7-417a-a5db-c0b8af49e4e4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:06.557773 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228da9-35d7-417a-a5db-c0b8af49e4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:06.558179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8228da9-35d7-417a-a5db-c0b8af49e4e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:06.558961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862581c5-606b-438d-9ffc-e5655322071f None admin] RBAC: Authorizing `identity:get_user(user_id=6b67c397b28b4ee4b1f4c3bd58f2c239)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:06.559860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8228da9-35d7-417a-a5db-c0b8af49e4e4 None admin] RBAC: Adding query filter params (role.id=0f01d78323f24797b5b1f2c555ea4c4c, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b, user.id=2ebde5c0c4a84150bc51fb6358ac5e4b) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:06.560329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8228da9-35d7-417a-a5db-c0b8af49e4e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:06.560739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862581c5-606b-438d-9ffc-e5655322071f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:06.562053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8228da9-35d7-417a-a5db-c0b8af49e4e4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:06.567282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 129/250] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:06 2022] GET /identity/v3/users/6b67c397b28b4ee4b1f4c3bd58f2c239 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:06.568560 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862581c5-606b-438d-9ffc-e5655322071f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:06.570473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 122/251] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:06 2022] GET /identity/v3/role_assignments?user.id=2ebde5c0c4a84150bc51fb6358ac5e4b&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=0f01d78323f24797b5b1f2c555ea4c4c => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:06.663849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c52206-999b-4da9-afab-d831adb571b5 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:06.664452 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c52206-999b-4da9-afab-d831adb571b5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:06.664925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c52206-999b-4da9-afab-d831adb571b5 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:06.666491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 123/252] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:06 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:41:06.671334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:06.673041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4353a-8d70-4d2e-8dea-867101e41e72 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:06.673332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4353a-8d70-4d2e-8dea-867101e41e72 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:06.673727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4353a-8d70-4d2e-8dea-867101e41e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:06.688105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862581c5-606b-438d-9ffc-e5655322071f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:06.702513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-63a4353a-8d70-4d2e-8dea-867101e41e72 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:06.704990 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-63a4353a-8d70-4d2e-8dea-867101e41e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:06.735110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd31eb-ea78-452f-a839-72cafdcfa27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:06.735772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd31eb-ea78-452f-a839-72cafdcfa27a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:06.736046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd31eb-ea78-452f-a839-72cafdcfa27a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:06.736318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd31eb-ea78-452f-a839-72cafdcfa27a None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:06.737512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd31eb-ea78-452f-a839-72cafdcfa27a None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:06.739869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bd31eb-ea78-452f-a839-72cafdcfa27a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:06.743051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 130/253] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:06 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:06.748025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd31eb-ea78-452f-a839-72cafdcfa27a None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:06.814537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 124/254] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:06.868133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd31eb-ea78-452f-a839-72cafdcfa27a None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:06.906095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2334d4-4642-42dc-a57c-5180b7be6416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:06.906763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2334d4-4642-42dc-a57c-5180b7be6416 None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:06.911116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2334d4-4642-42dc-a57c-5180b7be6416 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:06.911116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2334d4-4642-42dc-a57c-5180b7be6416 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/users/6b67c397b28b4ee4b1f4c3bd58f2c239/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:06.913112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2334d4-4642-42dc-a57c-5180b7be6416 None admin] RBAC: Authorizing `identity:create_grant(project_id=ff12bdfc9e9745779c7f414a2d09d14b, user_id=6b67c397b28b4ee4b1f4c3bd58f2c239, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:06.914676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2334d4-4642-42dc-a57c-5180b7be6416 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:07.002040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 131/255] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:41:06 2022] PUT /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/users/6b67c397b28b4ee4b1f4c3bd58f2c239/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:41:07.117344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4353a-8d70-4d2e-8dea-867101e41e72 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:07.298829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:07.328429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4353a-8d70-4d2e-8dea-867101e41e72 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:07.367629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e5696-8c19-4263-bfe5-9e1862de24db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:07.368507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e5696-8c19-4263-bfe5-9e1862de24db None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:07.368929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e5696-8c19-4263-bfe5-9e1862de24db None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:07.369465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e5696-8c19-4263-bfe5-9e1862de24db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:07.383675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e5696-8c19-4263-bfe5-9e1862de24db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-176e5696-8c19-4263-bfe5-9e1862de24db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:07.389541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:07.390620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:07.391666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:07.393852 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 125/256] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:41:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:07.399687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2334d4-4642-42dc-a57c-5180b7be6416 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:07.430274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:07.541987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2334d4-4642-42dc-a57c-5180b7be6416 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:07.560942 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c816812-879b-49c0-9edf-8216cab5a3ad None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:07.561871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c816812-879b-49c0-9edf-8216cab5a3ad None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:07.562338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c816812-879b-49c0-9edf-8216cab5a3ad None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:07.564428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 126/257] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:07 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:07.569881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420e423-022c-4c27-94b3-97efbd6eb12e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:07.570396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420e423-022c-4c27-94b3-97efbd6eb12e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:07.570864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420e423-022c-4c27-94b3-97efbd6eb12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:07.590976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8420e423-022c-4c27-94b3-97efbd6eb12e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:07.592586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8420e423-022c-4c27-94b3-97efbd6eb12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:07.594976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bf8b3-62a7-432d-861c-fd79dfdfed4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:07.595629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bf8b3-62a7-432d-861c-fd79dfdfed4a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:07.595907 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bf8b3-62a7-432d-861c-fd79dfdfed4a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:07.596896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6bf8b3-62a7-432d-861c-fd79dfdfed4a None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:07.596896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bf8b3-62a7-432d-861c-fd79dfdfed4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:07.597159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bf8b3-62a7-432d-861c-fd79dfdfed4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:07.598501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6bf8b3-62a7-432d-861c-fd79dfdfed4a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:07.617203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 132/258] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:07 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:07.629223 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bf8b3-62a7-432d-861c-fd79dfdfed4a None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:07.726171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 127/259] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:07.809601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:07.838592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6bf8b3-62a7-432d-861c-fd79dfdfed4a None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:07.881564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2499e-242b-46c9-a70c-6e99724c728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:07.882202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2499e-242b-46c9-a70c-6e99724c728c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:07.882483 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2499e-242b-46c9-a70c-6e99724c728c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:07.882760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2499e-242b-46c9-a70c-6e99724c728c None admin] PATH_INFO: `/v3/users/da8201ad76194976a6ee4942a0e057ca` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:07.883953 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2499e-242b-46c9-a70c-6e99724c728c None admin] RBAC: Authorizing `identity:get_user(user_id=da8201ad76194976a6ee4942a0e057ca)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:07.885396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2499e-242b-46c9-a70c-6e99724c728c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:07.888708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 133/260] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:07 2022] GET /identity/v3/users/da8201ad76194976a6ee4942a0e057ca => generated 284 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:07.893660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420e423-022c-4c27-94b3-97efbd6eb12e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:07.994350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:08.035900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420e423-022c-4c27-94b3-97efbd6eb12e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:08.088678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774b21d-d2db-475d-bb19-ca32fa137b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:08.089239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774b21d-d2db-475d-bb19-ca32fa137b55 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:08.089525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774b21d-d2db-475d-bb19-ca32fa137b55 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:08.089802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774b21d-d2db-475d-bb19-ca32fa137b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:08.182878 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774b21d-d2db-475d-bb19-ca32fa137b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:08.184972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774b21d-d2db-475d-bb19-ca32fa137b55 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:08.226492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2499e-242b-46c9-a70c-6e99724c728c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:08.297179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:08.372096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 128/261] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:07 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:08.373482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774b21d-d2db-475d-bb19-ca32fa137b55 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:08.417918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2499e-242b-46c9-a70c-6e99724c728c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:08.476282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ed7b2-b6ea-47ca-99ae-e7f972c3f52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:08.477049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ed7b2-b6ea-47ca-99ae-e7f972c3f52d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:08.477340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ed7b2-b6ea-47ca-99ae-e7f972c3f52d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:08.477776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ed7b2-b6ea-47ca-99ae-e7f972c3f52d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:08.489038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ed7b2-b6ea-47ca-99ae-e7f972c3f52d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-6b1ed7b2-b6ea-47ca-99ae-e7f972c3f52d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:08.494865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:08.495858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:08.496661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 134/262] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:41:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:08.502842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ed7b2-b6ea-47ca-99ae-e7f972c3f52d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:08.512601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:08.581985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774b21d-d2db-475d-bb19-ca32fa137b55 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:08.629259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34361b8e-b0b1-4c4b-a80b-a12367af713d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:08.630441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34361b8e-b0b1-4c4b-a80b-a12367af713d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:08.631559 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34361b8e-b0b1-4c4b-a80b-a12367af713d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:08.632196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34361b8e-b0b1-4c4b-a80b-a12367af713d None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:08.635324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34361b8e-b0b1-4c4b-a80b-a12367af713d None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:08.637195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34361b8e-b0b1-4c4b-a80b-a12367af713d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:08.640013 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 129/263] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:08 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:08.648439 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34361b8e-b0b1-4c4b-a80b-a12367af713d None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:08.651119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ed7b2-b6ea-47ca-99ae-e7f972c3f52d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:08.728375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5f59-786c-4e9f-b664-8c43059d210c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:08.729001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5f59-786c-4e9f-b664-8c43059d210c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:08.729281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5f59-786c-4e9f-b664-8c43059d210c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:08.729555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5f59-786c-4e9f-b664-8c43059d210c None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:08.730230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f5f59-786c-4e9f-b664-8c43059d210c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:08.730534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f5f59-786c-4e9f-b664-8c43059d210c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:08.732482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f5f59-786c-4e9f-b664-8c43059d210c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:08.743220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:08.753484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 135/264] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:08 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:08.762491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5f59-786c-4e9f-b664-8c43059d210c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:08.863796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34361b8e-b0b1-4c4b-a80b-a12367af713d None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:08.928918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795731e-fc6c-441f-b446-bba76685fdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:08.929456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795731e-fc6c-441f-b446-bba76685fdc7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:08.929736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795731e-fc6c-441f-b446-bba76685fdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:08.930004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c795731e-fc6c-441f-b446-bba76685fdc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:08.930347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:08.931395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795731e-fc6c-441f-b446-bba76685fdc7 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b, user.id=da8201ad76194976a6ee4942a0e057ca) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:08.931703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795731e-fc6c-441f-b446-bba76685fdc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:08.933143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c795731e-fc6c-441f-b446-bba76685fdc7 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:08.945302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 130/265] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:08 2022] GET /identity/v3/role_assignments?user.id=da8201ad76194976a6ee4942a0e057ca&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 610 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:09.003226 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5f59-786c-4e9f-b664-8c43059d210c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:09.060924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49644318-79e8-4e8b-af20-87bb3cc25c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:09.061563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49644318-79e8-4e8b-af20-87bb3cc25c05 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:09.061841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49644318-79e8-4e8b-af20-87bb3cc25c05 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:09.062113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49644318-79e8-4e8b-af20-87bb3cc25c05 None admin] PATH_INFO: `/v3/users/da8201ad76194976a6ee4942a0e057ca` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:09.063291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49644318-79e8-4e8b-af20-87bb3cc25c05 None admin] RBAC: Authorizing `identity:get_user(user_id=da8201ad76194976a6ee4942a0e057ca)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:09.064749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49644318-79e8-4e8b-af20-87bb3cc25c05 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:09.066333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 136/266] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:08 2022] GET /identity/v3/users/da8201ad76194976a6ee4942a0e057ca => generated 284 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:09.072358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795731e-fc6c-441f-b446-bba76685fdc7 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:09.247596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:09.251346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c795731e-fc6c-441f-b446-bba76685fdc7 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:09.293697 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e831b1-0752-478e-ac8d-63970bad9f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:09.294957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e831b1-0752-478e-ac8d-63970bad9f95 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:09.295286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e831b1-0752-478e-ac8d-63970bad9f95 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:09.295561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e831b1-0752-478e-ac8d-63970bad9f95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:09.397927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e831b1-0752-478e-ac8d-63970bad9f95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:09.399426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e831b1-0752-478e-ac8d-63970bad9f95 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:09.438526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459a876-5dc1-4236-9671-64ba8f6c6f29 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:09.438868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459a876-5dc1-4236-9671-64ba8f6c6f29 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:09.439115 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459a876-5dc1-4236-9671-64ba8f6c6f29 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:09.440773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 137/267] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:09.455671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1bc20-965b-4ea6-95d9-e7c56e33386f None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:09.455987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1bc20-965b-4ea6-95d9-e7c56e33386f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:09.456223 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1bc20-965b-4ea6-95d9-e7c56e33386f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:09.483449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-84d1bc20-965b-4ea6-95d9-e7c56e33386f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:09.484831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-84d1bc20-965b-4ea6-95d9-e7c56e33386f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:09.524605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:09.531303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 131/268] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:09.532653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977cfb0-b2a0-4412-a091-1ab4f7358cf9 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:09.532941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977cfb0-b2a0-4412-a091-1ab4f7358cf9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:09.533689 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977cfb0-b2a0-4412-a091-1ab4f7358cf9 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:09.535074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 132/269] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:09.540446 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977cfb0-b2a0-4412-a091-1ab4f7358cf9 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:09.583150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 138/270] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:09.584560 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f3a84-06ff-4da6-91a3-0ba7314d8b21 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:09.584850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f3a84-06ff-4da6-91a3-0ba7314d8b21 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:09.585123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f3a84-06ff-4da6-91a3-0ba7314d8b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:09.614924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ba9f3a84-06ff-4da6-91a3-0ba7314d8b21 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:09.616877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ba9f3a84-06ff-4da6-91a3-0ba7314d8b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:09.706195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 139/271] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:09.715064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:09.719650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977cfb0-b2a0-4412-a091-1ab4f7358cf9 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:09.735225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d45b15-a4bd-43c6-8aed-e2d4e63ca24b None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:09.735504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d45b15-a4bd-43c6-8aed-e2d4e63ca24b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:09.735784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d45b15-a4bd-43c6-8aed-e2d4e63ca24b None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:09.740905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 140/272] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:09 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:09.749109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3133b-7a84-4690-a3fc-415e44231f67 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:09.749396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3133b-7a84-4690-a3fc-415e44231f67 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:09.749673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec3133b-7a84-4690-a3fc-415e44231f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:09.770335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a41be-1aaa-42e3-95cd-8d73a8a66656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:09.771267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a41be-1aaa-42e3-95cd-8d73a8a66656 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:09.771694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a41be-1aaa-42e3-95cd-8d73a8a66656 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:09.772110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a41be-1aaa-42e3-95cd-8d73a8a66656 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:09.773851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a41be-1aaa-42e3-95cd-8d73a8a66656 None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:09.775542 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a41be-1aaa-42e3-95cd-8d73a8a66656 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:09.777952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 133/273] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:09 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:09.783440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a41be-1aaa-42e3-95cd-8d73a8a66656 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:09.791939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-0ec3133b-7a84-4690-a3fc-415e44231f67 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:09.793918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0ec3133b-7a84-4690-a3fc-415e44231f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:09.889497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 141/274] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:09.907970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:09.969545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a41be-1aaa-42e3-95cd-8d73a8a66656 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.004335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957820e-7a08-4591-85a7-e7f04bdc2c43 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.004839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957820e-7a08-4591-85a7-e7f04bdc2c43 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.005203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957820e-7a08-4591-85a7-e7f04bdc2c43 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.006364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 142/275] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:41:10.007381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957820e-7a08-4591-85a7-e7f04bdc2c43 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:10.016719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382561f-62c0-4e8c-a0d8-c048e8d592d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:10.017332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382561f-62c0-4e8c-a0d8-c048e8d592d5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.017659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382561f-62c0-4e8c-a0d8-c048e8d592d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.017934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382561f-62c0-4e8c-a0d8-c048e8d592d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.018920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382561f-62c0-4e8c-a0d8-c048e8d592d5 None admin] RBAC: Adding query filter params (role.id=b7f17471d2c948efa07d02424da68bea, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b, user.id=da8201ad76194976a6ee4942a0e057ca) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:10.018920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382561f-62c0-4e8c-a0d8-c048e8d592d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:10.020418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382561f-62c0-4e8c-a0d8-c048e8d592d5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:10.026847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 134/276] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:09 2022] GET /identity/v3/role_assignments?user.id=da8201ad76194976a6ee4942a0e057ca&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=b7f17471d2c948efa07d02424da68bea => generated 610 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:10.028091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f23ac-bd2e-4dca-b2d2-687f16c86467 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.032605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f23ac-bd2e-4dca-b2d2-687f16c86467 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.032605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f23ac-bd2e-4dca-b2d2-687f16c86467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.055279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-a13f23ac-bd2e-4dca-b2d2-687f16c86467 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:10.056883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a13f23ac-bd2e-4dca-b2d2-687f16c86467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:10.129473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957820e-7a08-4591-85a7-e7f04bdc2c43 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.143199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 135/277] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:10.144533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f23ac-bd2e-4dca-b2d2-687f16c86467 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:10.169457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1ec56-4730-4ed9-847c-a8bd124e4b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:10.171288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1ec56-4730-4ed9-847c-a8bd124e4b96 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.171784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1ec56-4730-4ed9-847c-a8bd124e4b96 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.172258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1ec56-4730-4ed9-847c-a8bd124e4b96 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.175378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:10.195029 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1ec56-4730-4ed9-847c-a8bd124e4b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-18b1ec56-4730-4ed9-847c-a8bd124e4b96 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:10.196057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:10.196904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:10.197699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 143/278] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:10.202882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1ec56-4730-4ed9-847c-a8bd124e4b96 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:10.282340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:10.325689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1ec56-4730-4ed9-847c-a8bd124e4b96 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.335392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f23ac-bd2e-4dca-b2d2-687f16c86467 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.362767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadec09-66c7-4ec5-83ca-29ff1650139f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:10.363448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadec09-66c7-4ec5-83ca-29ff1650139f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.363728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadec09-66c7-4ec5-83ca-29ff1650139f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.364004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadec09-66c7-4ec5-83ca-29ff1650139f None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.365107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadec09-66c7-4ec5-83ca-29ff1650139f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:10.366695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dadec09-66c7-4ec5-83ca-29ff1650139f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:10.368450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 144/279] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:10 2022] GET /identity/v3/domains/default => generated 200 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:10.369044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadec09-66c7-4ec5-83ca-29ff1650139f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:10.374100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ad58f-e607-4790-bbb6-d6de47ce00cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:10.374962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ad58f-e607-4790-bbb6-d6de47ce00cd None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.375416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ad58f-e607-4790-bbb6-d6de47ce00cd None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.375682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ad58f-e607-4790-bbb6-d6de47ce00cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.388219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ad58f-e607-4790-bbb6-d6de47ce00cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-be9ad58f-e607-4790-bbb6-d6de47ce00cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:10.393593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:10.394625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 136/280] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:10.395630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ad58f-e607-4790-bbb6-d6de47ce00cd None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:10.433458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:10.479421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadec09-66c7-4ec5-83ca-29ff1650139f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.494549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ad58f-e607-4790-bbb6-d6de47ce00cd None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.512629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76ee62-97f6-4b4c-a3cf-de8f80538397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:10.513227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76ee62-97f6-4b4c-a3cf-de8f80538397 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.513503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76ee62-97f6-4b4c-a3cf-de8f80538397 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.513789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76ee62-97f6-4b4c-a3cf-de8f80538397 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.514693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:10.514986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76ee62-97f6-4b4c-a3cf-de8f80538397 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:10.515313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76ee62-97f6-4b4c-a3cf-de8f80538397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:10.516670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76ee62-97f6-4b4c-a3cf-de8f80538397 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:10.524587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 145/281] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:10 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:10.527140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76ee62-97f6-4b4c-a3cf-de8f80538397 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:10.532744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2680c-25f3-426a-8424-70a538332fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:10.533292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2680c-25f3-426a-8424-70a538332fdf None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.533569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2680c-25f3-426a-8424-70a538332fdf None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.533840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2680c-25f3-426a-8424-70a538332fdf None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.534482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2680c-25f3-426a-8424-70a538332fdf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:10.535953 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb2680c-25f3-426a-8424-70a538332fdf None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:10.620926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76ee62-97f6-4b4c-a3cf-de8f80538397 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.631747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 137/282] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:41:10 2022] POST /identity/v3/users => generated 294 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:41:10.632572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2680c-25f3-426a-8424-70a538332fdf None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:10.635546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:10.658212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42245ea0-ec3c-440e-a784-7499837fef57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:10.658845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42245ea0-ec3c-440e-a784-7499837fef57 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.659233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42245ea0-ec3c-440e-a784-7499837fef57 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.659477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42245ea0-ec3c-440e-a784-7499837fef57 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.664583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42245ea0-ec3c-440e-a784-7499837fef57 None admin] RBAC: Adding query filter params (name=member) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:10.664583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42245ea0-ec3c-440e-a784-7499837fef57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:10.664583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42245ea0-ec3c-440e-a784-7499837fef57 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:10.677410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 146/283] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:10 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:10.677410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42245ea0-ec3c-440e-a784-7499837fef57 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:10.757786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2680c-25f3-426a-8424-70a538332fdf None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.789063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42245ea0-ec3c-440e-a784-7499837fef57 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.800076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535cbd3-2173-4c98-b35b-9d1f09f1616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:10.800721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535cbd3-2173-4c98-b35b-9d1f09f1616c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.801001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535cbd3-2173-4c98-b35b-9d1f09f1616c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.801336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535cbd3-2173-4c98-b35b-9d1f09f1616c None admin] PATH_INFO: `/v3/roles/member` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.815839 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535cbd3-2173-4c98-b35b-9d1f09f1616c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-b535cbd3-2173-4c98-b35b-9d1f09f1616c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:10.821540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:10.822435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:10.823254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 138/284] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:10.824350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535cbd3-2173-4c98-b35b-9d1f09f1616c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:10.834817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72851a8c-7711-4fc4-9bc2-67c285f5f6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:10.835558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72851a8c-7711-4fc4-9bc2-67c285f5f6e6 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.835993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72851a8c-7711-4fc4-9bc2-67c285f5f6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:10.836173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72851a8c-7711-4fc4-9bc2-67c285f5f6e6 None admin] PATH_INFO: `/v3/users/2ebde5c0c4a84150bc51fb6358ac5e4b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:10.837315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72851a8c-7711-4fc4-9bc2-67c285f5f6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=2ebde5c0c4a84150bc51fb6358ac5e4b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:10.838756 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72851a8c-7711-4fc4-9bc2-67c285f5f6e6 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:10.840367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 147/285] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:10 2022] GET /identity/v3/users/2ebde5c0c4a84150bc51fb6358ac5e4b => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:10.841121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72851a8c-7711-4fc4-9bc2-67c285f5f6e6 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:10.867119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:10.905094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:10.961157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535cbd3-2173-4c98-b35b-9d1f09f1616c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.969887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72851a8c-7711-4fc4-9bc2-67c285f5f6e6 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:10.998394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8d415-025a-45cd-a037-b5a0cc070bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:10.999611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8d415-025a-45cd-a037-b5a0cc070bda None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:10.999975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8d415-025a-45cd-a037-b5a0cc070bda None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:11.000250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8d415-025a-45cd-a037-b5a0cc070bda None admin] PATH_INFO: `/v3/users/b62babf337c2497ebace81b4e375221c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:11.014771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8d415-025a-45cd-a037-b5a0cc070bda None admin] RBAC: Authorizing `identity:get_user(user_id=b62babf337c2497ebace81b4e375221c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:11.016440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8d415-025a-45cd-a037-b5a0cc070bda None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:11.018398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 139/286] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:10 2022] GET /identity/v3/users/b62babf337c2497ebace81b4e375221c => generated 294 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:11.019204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8d415-025a-45cd-a037-b5a0cc070bda None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:11.025023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c646725-39e1-4e6b-ac82-38fc5335832c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:11.025656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c646725-39e1-4e6b-ac82-38fc5335832c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:11.025933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c646725-39e1-4e6b-ac82-38fc5335832c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:11.026205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c646725-39e1-4e6b-ac82-38fc5335832c None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:11.026899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c646725-39e1-4e6b-ac82-38fc5335832c None admin] RBAC: Adding query filter params (name=member) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:11.027227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c646725-39e1-4e6b-ac82-38fc5335832c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:11.028574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c646725-39e1-4e6b-ac82-38fc5335832c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:11.042922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 148/287] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:10 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:11.043736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c646725-39e1-4e6b-ac82-38fc5335832c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:11.129580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:11.136486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8d415-025a-45cd-a037-b5a0cc070bda None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:11.174498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33db4d-e27f-42ff-af0a-518b6856541a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:11.175427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33db4d-e27f-42ff-af0a-518b6856541a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:11.176032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33db4d-e27f-42ff-af0a-518b6856541a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:11.176311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33db4d-e27f-42ff-af0a-518b6856541a None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:11.178427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33db4d-e27f-42ff-af0a-518b6856541a None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:11.180189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33db4d-e27f-42ff-af0a-518b6856541a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:11.180904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c646725-39e1-4e6b-ac82-38fc5335832c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:11.185956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 140/288] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:11 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:11.188012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33db4d-e27f-42ff-af0a-518b6856541a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:11.194509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:11.223057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16757a-f579-496b-bfad-6cf694223492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:11.223969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16757a-f579-496b-bfad-6cf694223492 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:11.224449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16757a-f579-496b-bfad-6cf694223492 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:11.224895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16757a-f579-496b-bfad-6cf694223492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:11.225808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16757a-f579-496b-bfad-6cf694223492 None admin] RBAC: Adding query filter params (role.id=df27fa65390e4de78fa117773dc73555, user.id=b62babf337c2497ebace81b4e375221c, scope.system=all) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:11.226288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16757a-f579-496b-bfad-6cf694223492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:11.227946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16757a-f579-496b-bfad-6cf694223492 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:11.235570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 149/289] 10.222.0.105 () {56 vars in 1313 bytes} [Wed Aug 24 04:41:11 2022] GET /identity/v3/role_assignments?user.id=b62babf337c2497ebace81b4e375221c&scope.system=all&role.id=df27fa65390e4de78fa117773dc73555 => generated 227 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:11.236659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16757a-f579-496b-bfad-6cf694223492 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:11.326024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33db4d-e27f-42ff-af0a-518b6856541a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:11.370540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c3161-659d-4f62-855b-1e75813f5f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:11.371532 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c3161-659d-4f62-855b-1e75813f5f04 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:11.371811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c3161-659d-4f62-855b-1e75813f5f04 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:11.372082 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c3161-659d-4f62-855b-1e75813f5f04 None admin] PATH_INFO: `/v3/users/6b67c397b28b4ee4b1f4c3bd58f2c239` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:11.373256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c3161-659d-4f62-855b-1e75813f5f04 None admin] RBAC: Authorizing `identity:get_user(user_id=6b67c397b28b4ee4b1f4c3bd58f2c239)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:11.375792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c3161-659d-4f62-855b-1e75813f5f04 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:11.377918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 141/290] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:11 2022] GET /identity/v3/users/6b67c397b28b4ee4b1f4c3bd58f2c239 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:11.383615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c3161-659d-4f62-855b-1e75813f5f04 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:11.422994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:11.470045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16757a-f579-496b-bfad-6cf694223492 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:11.504477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c3161-659d-4f62-855b-1e75813f5f04 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:11.540998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f58ac-c8eb-4bc7-aee0-cddf378b66a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:11.541838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f58ac-c8eb-4bc7-aee0-cddf378b66a6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:11.542289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f58ac-c8eb-4bc7-aee0-cddf378b66a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:11.542700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f58ac-c8eb-4bc7-aee0-cddf378b66a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:11.588683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749dced-b5b7-4a27-a6b9-40587863872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:11.589462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749dced-b5b7-4a27-a6b9-40587863872d None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:11.589579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749dced-b5b7-4a27-a6b9-40587863872d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:11.589863 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749dced-b5b7-4a27-a6b9-40587863872d None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/users/2ebde5c0c4a84150bc51fb6358ac5e4b/roles/0f01d78323f24797b5b1f2c555ea4c4c` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:11.599166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749dced-b5b7-4a27-a6b9-40587863872d None admin] RBAC: Authorizing `identity:create_grant(project_id=ff12bdfc9e9745779c7f414a2d09d14b, user_id=2ebde5c0c4a84150bc51fb6358ac5e4b, role_id=0f01d78323f24797b5b1f2c555ea4c4c)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:11.600619 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9749dced-b5b7-4a27-a6b9-40587863872d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:11.616518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f58ac-c8eb-4bc7-aee0-cddf378b66a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:11.618242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f58ac-c8eb-4bc7-aee0-cddf378b66a6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:11.661953 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:11.717357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 150/291] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:41:11 2022] PUT /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/users/2ebde5c0c4a84150bc51fb6358ac5e4b/roles/0f01d78323f24797b5b1f2c555ea4c4c => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:11.721554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 142/292] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:11 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:11.727524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749dced-b5b7-4a27-a6b9-40587863872d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:11.762615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:11.837596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749dced-b5b7-4a27-a6b9-40587863872d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:11.874901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d828723a-387e-4d2c-8700-7c7bfad09e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:11.875881 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828723a-387e-4d2c-8700-7c7bfad09e50 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:11.876766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828723a-387e-4d2c-8700-7c7bfad09e50 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:11.877185 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d828723a-387e-4d2c-8700-7c7bfad09e50 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:11.878594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828723a-387e-4d2c-8700-7c7bfad09e50 None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:11.880277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d828723a-387e-4d2c-8700-7c7bfad09e50 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:11.882524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 151/293] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:11 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:11.888614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f58ac-c8eb-4bc7-aee0-cddf378b66a6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:12.002127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f58ac-c8eb-4bc7-aee0-cddf378b66a6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:12.032652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:12.038579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3677b0e-ac15-4d06-af9c-0ef71591856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:12.039410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3677b0e-ac15-4d06-af9c-0ef71591856e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:12.039832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3677b0e-ac15-4d06-af9c-0ef71591856e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:12.040246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3677b0e-ac15-4d06-af9c-0ef71591856e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:12.041112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3677b0e-ac15-4d06-af9c-0ef71591856e None admin] RBAC: Adding query filter params (role.id=df27fa65390e4de78fa117773dc73555, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b, user.id=6b67c397b28b4ee4b1f4c3bd58f2c239) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:12.041577 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3677b0e-ac15-4d06-af9c-0ef71591856e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:12.043175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3677b0e-ac15-4d06-af9c-0ef71591856e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:12.050062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 143/294] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:11 2022] GET /identity/v3/role_assignments?user.id=6b67c397b28b4ee4b1f4c3bd58f2c239&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=df27fa65390e4de78fa117773dc73555 => generated 610 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:12.835758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd175-f726-40c5-a359-80995c6bc521 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:12.836292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd175-f726-40c5-a359-80995c6bc521 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:12.836292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fd175-f726-40c5-a359-80995c6bc521 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:12.837858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 152/295] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:12.843776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1f098-c62f-48d1-9d6c-3d87d95ac801 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:12.844314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1f098-c62f-48d1-9d6c-3d87d95ac801 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:12.844794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1f098-c62f-48d1-9d6c-3d87d95ac801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:12.864921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-a8d1f098-c62f-48d1-9d6c-3d87d95ac801 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:12.866341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a8d1f098-c62f-48d1-9d6c-3d87d95ac801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:12.892225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30357359-bc6a-498c-a265-0f395cfc99a2 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:12.892838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30357359-bc6a-498c-a265-0f395cfc99a2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:12.893313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30357359-bc6a-498c-a265-0f395cfc99a2 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:12.896300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 153/296] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:12 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:12.900621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ffc00-d822-4e30-a9ec-bd14cc800c7e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:12.900913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ffc00-d822-4e30-a9ec-bd14cc800c7e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:12.901187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ffc00-d822-4e30-a9ec-bd14cc800c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:12.919544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-e28ffc00-d822-4e30-a9ec-bd14cc800c7e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:12.920736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e28ffc00-d822-4e30-a9ec-bd14cc800c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:12.941747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 144/297] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:12.994979 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 154/298] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:12 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:13.223581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1f098-c62f-48d1-9d6c-3d87d95ac801 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:13.283844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ffc00-d822-4e30-a9ec-bd14cc800c7e None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:13.319432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:13.335096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1f098-c62f-48d1-9d6c-3d87d95ac801 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:13.365534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:13.378749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7d347-4268-4635-8357-84ebfe624639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:13.380783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7d347-4268-4635-8357-84ebfe624639 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:13.381072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7d347-4268-4635-8357-84ebfe624639 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:13.381343 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b7d347-4268-4635-8357-84ebfe624639 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:13.394541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ffc00-d822-4e30-a9ec-bd14cc800c7e None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:13.395394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b7d347-4268-4635-8357-84ebfe624639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-32b7d347-4268-4635-8357-84ebfe624639 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:13.401599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:13.402511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:13.403798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:13.407123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 145/299] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:13.409509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7d347-4268-4635-8357-84ebfe624639 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:13.442913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0df91-5eaf-4fc3-9e5f-a34d70737f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:13.443515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0df91-5eaf-4fc3-9e5f-a34d70737f5d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:13.443845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0df91-5eaf-4fc3-9e5f-a34d70737f5d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:13.444116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0df91-5eaf-4fc3-9e5f-a34d70737f5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:13.445216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0df91-5eaf-4fc3-9e5f-a34d70737f5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:13.446828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0df91-5eaf-4fc3-9e5f-a34d70737f5d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:13.453392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 155/300] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:13 2022] GET /identity/v3/domains/default => generated 200 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:13.454685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0df91-5eaf-4fc3-9e5f-a34d70737f5d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:13.530463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b7d347-4268-4635-8357-84ebfe624639 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:13.562887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:13.567441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a55db-5ca6-4782-9ef7-02bc024aa2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:13.568132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a55db-5ca6-4782-9ef7-02bc024aa2a5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:13.568403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a55db-5ca6-4782-9ef7-02bc024aa2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:13.568667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a55db-5ca6-4782-9ef7-02bc024aa2a5 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:13.569372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0df91-5eaf-4fc3-9e5f-a34d70737f5d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:13.569468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a55db-5ca6-4782-9ef7-02bc024aa2a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:13.569671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a55db-5ca6-4782-9ef7-02bc024aa2a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:13.571043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a55db-5ca6-4782-9ef7-02bc024aa2a5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:13.580027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 146/301] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:13 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:13.585006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943f214-37bf-447c-ab7c-01669cdb50c7 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:13.585293 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943f214-37bf-447c-ab7c-01669cdb50c7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:13.585705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943f214-37bf-447c-ab7c-01669cdb50c7 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:13.587109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 147/302] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:13.591696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943f214-37bf-447c-ab7c-01669cdb50c7 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:13.607136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6cc2ad-53e5-453b-a80f-01624524b197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:13.607847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc2ad-53e5-453b-a80f-01624524b197 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:13.608123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc2ad-53e5-453b-a80f-01624524b197 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:13.608417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6cc2ad-53e5-453b-a80f-01624524b197 None admin] PATH_INFO: `/v3/groups` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:13.609051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cc2ad-53e5-453b-a80f-01624524b197 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:13.611258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6cc2ad-53e5-453b-a80f-01624524b197 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:13.639963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:13.670046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 156/303] 10.222.0.105 () {60 vars in 1139 bytes} [Wed Aug 24 04:41:13 2022] POST /identity/v3/groups => generated 230 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:41:13.681401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e707919-e974-4738-9e26-11d521edc318 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:13.681754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e707919-e974-4738-9e26-11d521edc318 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:13.681958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e707919-e974-4738-9e26-11d521edc318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:13.701584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943f214-37bf-447c-ab7c-01669cdb50c7 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:13.708649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2e707919-e974-4738-9e26-11d521edc318 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:13.709867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2e707919-e974-4738-9e26-11d521edc318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:13.744475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60ab35-f5b2-47b3-9671-7c5057338e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:13.745484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60ab35-f5b2-47b3-9671-7c5057338e1a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:13.745974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60ab35-f5b2-47b3-9671-7c5057338e1a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:13.746514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60ab35-f5b2-47b3-9671-7c5057338e1a None admin] PATH_INFO: `/v3/users/8b2928fe42994d2d92ab0d7b2d23bda0` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:13.763213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60ab35-f5b2-47b3-9671-7c5057338e1a None admin] RBAC: Authorizing `identity:get_user(user_id=8b2928fe42994d2d92ab0d7b2d23bda0)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:13.765271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60ab35-f5b2-47b3-9671-7c5057338e1a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:13.773211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 148/304] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:13 2022] GET /identity/v3/users/8b2928fe42994d2d92ab0d7b2d23bda0 => generated 294 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:13.774970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60ab35-f5b2-47b3-9671-7c5057338e1a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:13.806812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 157/305] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:13.886510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60ab35-f5b2-47b3-9671-7c5057338e1a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:13.888994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:13.922561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a541a8-9617-436d-88b5-8d3b26342d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:13.922991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a541a8-9617-436d-88b5-8d3b26342d7f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:13.923293 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a541a8-9617-436d-88b5-8d3b26342d7f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:13.923556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a541a8-9617-436d-88b5-8d3b26342d7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:13.924329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a541a8-9617-436d-88b5-8d3b26342d7f None admin] RBAC: Adding query filter params (role.id=0f01d78323f24797b5b1f2c555ea4c4c, user.id=8b2928fe42994d2d92ab0d7b2d23bda0, scope.system=all) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:13.924616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a541a8-9617-436d-88b5-8d3b26342d7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:13.926050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a541a8-9617-436d-88b5-8d3b26342d7f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:13.932226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 149/306] 10.222.0.105 () {56 vars in 1313 bytes} [Wed Aug 24 04:41:13 2022] GET /identity/v3/role_assignments?user.id=8b2928fe42994d2d92ab0d7b2d23bda0&scope.system=all&role.id=0f01d78323f24797b5b1f2c555ea4c4c => generated 227 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:14.006479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c70c-afb2-492d-93d6-fa5ecd1f630b None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:14.006998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c70c-afb2-492d-93d6-fa5ecd1f630b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:14.006998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c70c-afb2-492d-93d6-fa5ecd1f630b None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:14.008713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 158/307] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:14.014919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238e4b7-257d-4fae-bd5b-822c82de141a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:14.015454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238e4b7-257d-4fae-bd5b-822c82de141a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:14.015528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3238e4b7-257d-4fae-bd5b-822c82de141a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:14.036297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-3238e4b7-257d-4fae-bd5b-822c82de141a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:14.037507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3238e4b7-257d-4fae-bd5b-822c82de141a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:14.087244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c70c-afb2-492d-93d6-fa5ecd1f630b None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:14.113117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 150/308] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:14.179407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:14.196074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c70c-afb2-492d-93d6-fa5ecd1f630b None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:14.231245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852ff5a-f2fb-4393-8c17-08f730bff33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:14.232079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ff5a-f2fb-4393-8c17-08f730bff33e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:14.232410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ff5a-f2fb-4393-8c17-08f730bff33e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:14.233026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852ff5a-f2fb-4393-8c17-08f730bff33e None admin] PATH_INFO: `/v3/roles/member` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:14.244195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852ff5a-f2fb-4393-8c17-08f730bff33e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b852ff5a-f2fb-4393-8c17-08f730bff33e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:14.248962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:14.249846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:14.251363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 159/309] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:14.256831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238e4b7-257d-4fae-bd5b-822c82de141a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:14.355173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:14.359405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3238e4b7-257d-4fae-bd5b-822c82de141a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:14.396942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852ff5a-f2fb-4393-8c17-08f730bff33e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:14.400382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae88ac3-d0b1-4ea8-a095-4aa83f05ec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:14.401005 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae88ac3-d0b1-4ea8-a095-4aa83f05ec1e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:14.401279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae88ac3-d0b1-4ea8-a095-4aa83f05ec1e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:14.401546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae88ac3-d0b1-4ea8-a095-4aa83f05ec1e None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:14.402204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae88ac3-d0b1-4ea8-a095-4aa83f05ec1e None admin] RBAC: Adding query filter params (name=member) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:14.402501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae88ac3-d0b1-4ea8-a095-4aa83f05ec1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:14.403888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae88ac3-d0b1-4ea8-a095-4aa83f05ec1e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:14.413861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 151/310] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:14 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:14.419510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae88ac3-d0b1-4ea8-a095-4aa83f05ec1e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:14.511143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852ff5a-f2fb-4393-8c17-08f730bff33e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:14.547538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4896df-54cb-4977-8c98-41c0ab647e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:14.548518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4896df-54cb-4977-8c98-41c0ab647e9f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:14.548993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4896df-54cb-4977-8c98-41c0ab647e9f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:14.549453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4896df-54cb-4977-8c98-41c0ab647e9f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:14.557796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae88ac3-d0b1-4ea8-a095-4aa83f05ec1e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:14.563525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4896df-54cb-4977-8c98-41c0ab647e9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-ee4896df-54cb-4977-8c98-41c0ab647e9f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:14.569828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:14.570689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:14.571526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:14.572699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 160/311] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:14.578310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4896df-54cb-4977-8c98-41c0ab647e9f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:14.588360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:14.600409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842e116-1b8a-4c54-8e82-1b16abe92148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:14.603197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842e116-1b8a-4c54-8e82-1b16abe92148 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:14.603197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842e116-1b8a-4c54-8e82-1b16abe92148 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:14.603197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842e116-1b8a-4c54-8e82-1b16abe92148 None admin] PATH_INFO: `/v3/users/b62babf337c2497ebace81b4e375221c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:14.603566 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842e116-1b8a-4c54-8e82-1b16abe92148 None admin] RBAC: Authorizing `identity:get_user(user_id=b62babf337c2497ebace81b4e375221c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:14.605180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842e116-1b8a-4c54-8e82-1b16abe92148 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:14.607278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 152/312] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:14 2022] GET /identity/v3/users/b62babf337c2497ebace81b4e375221c => generated 294 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:14.612801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842e116-1b8a-4c54-8e82-1b16abe92148 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:14.656095 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:14.694016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4896df-54cb-4977-8c98-41c0ab647e9f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:14.731638 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449800ef-6c5f-45d4-b5ff-3a87b513b264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:14.732645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449800ef-6c5f-45d4-b5ff-3a87b513b264 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:14.733115 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449800ef-6c5f-45d4-b5ff-3a87b513b264 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:14.733624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449800ef-6c5f-45d4-b5ff-3a87b513b264 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:14.734693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449800ef-6c5f-45d4-b5ff-3a87b513b264 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:14.735257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449800ef-6c5f-45d4-b5ff-3a87b513b264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:14.736886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449800ef-6c5f-45d4-b5ff-3a87b513b264 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:14.737454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842e116-1b8a-4c54-8e82-1b16abe92148 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:14.750757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 161/313] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:14 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:14.752558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449800ef-6c5f-45d4-b5ff-3a87b513b264 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:14.773338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4f4ef-4931-4105-8b53-a9235361e893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:14.774218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4f4ef-4931-4105-8b53-a9235361e893 None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:14.774502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4f4ef-4931-4105-8b53-a9235361e893 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:14.774775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4f4ef-4931-4105-8b53-a9235361e893 None admin] PATH_INFO: `/v3/system/users/b62babf337c2497ebace81b4e375221c/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:14.776234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4f4ef-4931-4105-8b53-a9235361e893 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b62babf337c2497ebace81b4e375221c, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:14.777593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4f4ef-4931-4105-8b53-a9235361e893 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:14.825675 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:14.858597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449800ef-6c5f-45d4-b5ff-3a87b513b264 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:14.875424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 153/314] 10.222.0.105 () {58 vars in 1339 bytes} [Wed Aug 24 04:41:14 2022] PUT /identity/v3/system/users/b62babf337c2497ebace81b4e375221c/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:14.904724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187ecb6-3152-4ae6-b21d-099c78c250fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:14.905380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187ecb6-3152-4ae6-b21d-099c78c250fc None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:14.905655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187ecb6-3152-4ae6-b21d-099c78c250fc None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:14.905923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187ecb6-3152-4ae6-b21d-099c78c250fc None admin] PATH_INFO: `/v3/users/2ebde5c0c4a84150bc51fb6358ac5e4b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:14.908164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187ecb6-3152-4ae6-b21d-099c78c250fc None admin] RBAC: Authorizing `identity:get_user(user_id=2ebde5c0c4a84150bc51fb6358ac5e4b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:14.910333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187ecb6-3152-4ae6-b21d-099c78c250fc None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:14.917414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 162/315] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:14 2022] GET /identity/v3/users/2ebde5c0c4a84150bc51fb6358ac5e4b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:14.919053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4f4ef-4931-4105-8b53-a9235361e893 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:15.032469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:15.035298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4f4ef-4931-4105-8b53-a9235361e893 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:15.071567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122d04e-9f20-4b45-9799-9ce2fab38f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:15.072179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122d04e-9f20-4b45-9799-9ce2fab38f1e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:15.072448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122d04e-9f20-4b45-9799-9ce2fab38f1e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:15.072709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122d04e-9f20-4b45-9799-9ce2fab38f1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:15.143912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122d04e-9f20-4b45-9799-9ce2fab38f1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:15.145554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122d04e-9f20-4b45-9799-9ce2fab38f1e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:15.239727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:15.249661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 154/316] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:14 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:15.256288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187ecb6-3152-4ae6-b21d-099c78c250fc None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:15.363351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187ecb6-3152-4ae6-b21d-099c78c250fc None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:15.400599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9d83a-7b1e-4933-835f-b8cb8034bc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:15.401232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9d83a-7b1e-4933-835f-b8cb8034bc1c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:15.401561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9d83a-7b1e-4933-835f-b8cb8034bc1c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:15.401829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9d83a-7b1e-4933-835f-b8cb8034bc1c None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:15.402957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9d83a-7b1e-4933-835f-b8cb8034bc1c None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:15.404419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9d83a-7b1e-4933-835f-b8cb8034bc1c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:15.406376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 163/317] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:15 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:15.412183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122d04e-9f20-4b45-9799-9ce2fab38f1e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:15.512832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122d04e-9f20-4b45-9799-9ce2fab38f1e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:15.552723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa93bc-7b53-4f95-8468-8328a706099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:15.553670 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa93bc-7b53-4f95-8468-8328a706099e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:15.554145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa93bc-7b53-4f95-8468-8328a706099e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:15.554903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa93bc-7b53-4f95-8468-8328a706099e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:15.555836 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa93bc-7b53-4f95-8468-8328a706099e None admin] RBAC: Adding query filter params (role.id=0f01d78323f24797b5b1f2c555ea4c4c, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b, user.id=2ebde5c0c4a84150bc51fb6358ac5e4b) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:15.556401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa93bc-7b53-4f95-8468-8328a706099e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:15.558016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa93bc-7b53-4f95-8468-8328a706099e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:15.565200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 155/318] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:15 2022] GET /identity/v3/role_assignments?user.id=2ebde5c0c4a84150bc51fb6358ac5e4b&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=0f01d78323f24797b5b1f2c555ea4c4c => generated 610 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:15.787495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49730b00-086c-4490-aa09-424ef87129a3 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:15.787833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49730b00-086c-4490-aa09-424ef87129a3 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:15.788005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49730b00-086c-4490-aa09-424ef87129a3 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:15.789455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 164/319] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:15.795179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba24a7a-8bdf-4823-a25d-6d24d3fb20f0 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:15.795743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba24a7a-8bdf-4823-a25d-6d24d3fb20f0 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:15.796208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba24a7a-8bdf-4823-a25d-6d24d3fb20f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:15.814833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6ba24a7a-8bdf-4823-a25d-6d24d3fb20f0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:15.816030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6ba24a7a-8bdf-4823-a25d-6d24d3fb20f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:15.867653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:15.883503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 156/320] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:15.932467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9ea7-af6b-4818-a90c-18de95d8e224 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:15.932771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9ea7-af6b-4818-a90c-18de95d8e224 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:15.933009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b9ea7-af6b-4818-a90c-18de95d8e224 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:15.934466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 165/321] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:15.940034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f9b68-3576-437c-ab7e-da040e3f6042 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:15.940324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f9b68-3576-437c-ab7e-da040e3f6042 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:15.940596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f9b68-3576-437c-ab7e-da040e3f6042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:15.959481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-293f9b68-3576-437c-ab7e-da040e3f6042 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:15.960658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-293f9b68-3576-437c-ab7e-da040e3f6042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:16.030497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 157/322] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:15 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:16.152859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b9ea7-af6b-4818-a90c-18de95d8e224 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:16.157096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:16.262477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b9ea7-af6b-4818-a90c-18de95d8e224 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:16.296850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457a8a3-ebc5-4a52-9ee6-798fe4fd4e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:16.297472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457a8a3-ebc5-4a52-9ee6-798fe4fd4e73 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:16.297750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457a8a3-ebc5-4a52-9ee6-798fe4fd4e73 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:16.298021 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457a8a3-ebc5-4a52-9ee6-798fe4fd4e73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:16.299336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457a8a3-ebc5-4a52-9ee6-798fe4fd4e73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:16.300764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457a8a3-ebc5-4a52-9ee6-798fe4fd4e73 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:16.302540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 166/323] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:16 2022] GET /identity/v3/domains/default => generated 200 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:16.308107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f9b68-3576-437c-ab7e-da040e3f6042 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:16.309424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457a8a3-ebc5-4a52-9ee6-798fe4fd4e73 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:16.396440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:16.415833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f9b68-3576-437c-ab7e-da040e3f6042 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:16.434671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:16.435165 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457a8a3-ebc5-4a52-9ee6-798fe4fd4e73 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:16.452016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a9422-ad9e-497f-a0be-51bd54212f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:16.452630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a9422-ad9e-497f-a0be-51bd54212f0f None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:16.452911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a9422-ad9e-497f-a0be-51bd54212f0f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:16.453184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a9422-ad9e-497f-a0be-51bd54212f0f None admin] PATH_INFO: `/v3/groups` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:16.453819 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a9422-ad9e-497f-a0be-51bd54212f0f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:16.455358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a9422-ad9e-497f-a0be-51bd54212f0f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:16.472833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab58602-f078-47c6-b13c-e3e8c6b4b7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:16.473552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab58602-f078-47c6-b13c-e3e8c6b4b7d8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:16.478971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab58602-f078-47c6-b13c-e3e8c6b4b7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:16.478971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab58602-f078-47c6-b13c-e3e8c6b4b7d8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:16.484571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab58602-f078-47c6-b13c-e3e8c6b4b7d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-9ab58602-f078-47c6-b13c-e3e8c6b4b7d8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:16.490525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:16.491636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:16.492548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 167/324] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:16.497320 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab58602-f078-47c6-b13c-e3e8c6b4b7d8 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:16.566946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 158/325] 10.222.0.105 () {60 vars in 1139 bytes} [Wed Aug 24 04:41:16 2022] POST /identity/v3/groups => generated 227 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:41:16.607456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab58602-f078-47c6-b13c-e3e8c6b4b7d8 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:16.641859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:16.648464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62b66e-82e1-4635-b755-bebe5c98a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:16.649074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62b66e-82e1-4635-b755-bebe5c98a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:16.649356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62b66e-82e1-4635-b755-bebe5c98a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:16.649632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62b66e-82e1-4635-b755-bebe5c98a7c3 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:16.650376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62b66e-82e1-4635-b755-bebe5c98a7c3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:16.650721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62b66e-82e1-4635-b755-bebe5c98a7c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:16.652202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62b66e-82e1-4635-b755-bebe5c98a7c3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:16.664199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 168/326] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:16 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:16.669802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a9422-ad9e-497f-a0be-51bd54212f0f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:16.778133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a9422-ad9e-497f-a0be-51bd54212f0f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:16.818607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e70eec-73b6-48e2-ad8e-5658ba75f405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:16.819395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e70eec-73b6-48e2-ad8e-5658ba75f405 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:16.819675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e70eec-73b6-48e2-ad8e-5658ba75f405 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:16.819946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e70eec-73b6-48e2-ad8e-5658ba75f405 None admin] PATH_INFO: `/v3/users/8b2928fe42994d2d92ab0d7b2d23bda0` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:16.821107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e70eec-73b6-48e2-ad8e-5658ba75f405 None admin] RBAC: Authorizing `identity:get_user(user_id=8b2928fe42994d2d92ab0d7b2d23bda0)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:16.822542 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e70eec-73b6-48e2-ad8e-5658ba75f405 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:16.824514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 159/327] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:16 2022] GET /identity/v3/users/8b2928fe42994d2d92ab0d7b2d23bda0 => generated 294 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:16.830906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62b66e-82e1-4635-b755-bebe5c98a7c3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:16.897990 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225784be-0649-4072-b396-d457dfa5a18a None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:16.898303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225784be-0649-4072-b396-d457dfa5a18a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:16.898540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225784be-0649-4072-b396-d457dfa5a18a None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:16.900190 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 160/328] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:41:16.905044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c663d-4692-4046-b83b-845d0f76f5cd None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:16.905327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c663d-4692-4046-b83b-845d0f76f5cd None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:16.905601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c663d-4692-4046-b83b-845d0f76f5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:16.925184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b62c663d-4692-4046-b83b-845d0f76f5cd None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:16.926292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b62c663d-4692-4046-b83b-845d0f76f5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:16.951814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62b66e-82e1-4635-b755-bebe5c98a7c3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:16.978587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:16.997822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb30ff0-aa74-4d02-8cd3-8d018263959c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:16.998485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb30ff0-aa74-4d02-8cd3-8d018263959c None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:16.998766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb30ff0-aa74-4d02-8cd3-8d018263959c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:16.999257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb30ff0-aa74-4d02-8cd3-8d018263959c None admin] PATH_INFO: `/v3/system/users/8b2928fe42994d2d92ab0d7b2d23bda0/roles/0f01d78323f24797b5b1f2c555ea4c4c` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:17.001060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb30ff0-aa74-4d02-8cd3-8d018263959c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8b2928fe42994d2d92ab0d7b2d23bda0, role_id=0f01d78323f24797b5b1f2c555ea4c4c)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:17.004347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb30ff0-aa74-4d02-8cd3-8d018263959c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:17.021561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 161/329] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:16 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:17.078078 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 169/330] 10.222.0.105 () {58 vars in 1339 bytes} [Wed Aug 24 04:41:16 2022] PUT /identity/v3/system/users/8b2928fe42994d2d92ab0d7b2d23bda0/roles/0f01d78323f24797b5b1f2c555ea4c4c => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:17.316651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c663d-4692-4046-b83b-845d0f76f5cd None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:17.424829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c663d-4692-4046-b83b-845d0f76f5cd None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:17.461757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd97ba-a2d6-4efb-8953-44e9656d542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:17.462367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd97ba-a2d6-4efb-8953-44e9656d542f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:17.462648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd97ba-a2d6-4efb-8953-44e9656d542f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:17.462911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd97ba-a2d6-4efb-8953-44e9656d542f None admin] PATH_INFO: `/v3/roles/member` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:17.475079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd97ba-a2d6-4efb-8953-44e9656d542f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-d5bd97ba-a2d6-4efb-8953-44e9656d542f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:17.479927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:17.480822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:17.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:17.485822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 162/331] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:17.487249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb30ff0-aa74-4d02-8cd3-8d018263959c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:17.554829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:17.603858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb30ff0-aa74-4d02-8cd3-8d018263959c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:17.641631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90f932-a7ad-4303-a8ed-8d3ce3199ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:17.642247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90f932-a7ad-4303-a8ed-8d3ce3199ca3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:17.642524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90f932-a7ad-4303-a8ed-8d3ce3199ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:17.642901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90f932-a7ad-4303-a8ed-8d3ce3199ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:17.643598 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90f932-a7ad-4303-a8ed-8d3ce3199ca3 None admin] RBAC: Adding query filter params (name=member) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:17.643897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90f932-a7ad-4303-a8ed-8d3ce3199ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:17.645261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90f932-a7ad-4303-a8ed-8d3ce3199ca3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:17.657133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 170/332] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:17 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:17.662846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd97ba-a2d6-4efb-8953-44e9656d542f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:17.701431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:17.791560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd97ba-a2d6-4efb-8953-44e9656d542f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:17.837711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e289152-c32c-4e5d-a49e-3ffbac2ccb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:17.838360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e289152-c32c-4e5d-a49e-3ffbac2ccb0f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:17.838640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e289152-c32c-4e5d-a49e-3ffbac2ccb0f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:17.838912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e289152-c32c-4e5d-a49e-3ffbac2ccb0f None admin] PATH_INFO: `/v3/users/b62babf337c2497ebace81b4e375221c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:17.840115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e289152-c32c-4e5d-a49e-3ffbac2ccb0f None admin] RBAC: Authorizing `identity:get_user(user_id=b62babf337c2497ebace81b4e375221c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:17.841564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e289152-c32c-4e5d-a49e-3ffbac2ccb0f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:17.843489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 163/333] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:17 2022] GET /identity/v3/users/b62babf337c2497ebace81b4e375221c => generated 294 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:17.850112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90f932-a7ad-4303-a8ed-8d3ce3199ca3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:17.982563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90f932-a7ad-4303-a8ed-8d3ce3199ca3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:18.003924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:18.022848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9552e-43c3-4ff5-825a-91b3bc9d45cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:18.023625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9552e-43c3-4ff5-825a-91b3bc9d45cf None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:18.023936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9552e-43c3-4ff5-825a-91b3bc9d45cf None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:18.024212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9552e-43c3-4ff5-825a-91b3bc9d45cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:18.025034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9552e-43c3-4ff5-825a-91b3bc9d45cf None admin] RBAC: Adding query filter params (role.id=df27fa65390e4de78fa117773dc73555, user.id=b62babf337c2497ebace81b4e375221c, scope.system=all) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:18.025338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9552e-43c3-4ff5-825a-91b3bc9d45cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:18.026841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9552e-43c3-4ff5-825a-91b3bc9d45cf None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:18.042621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 171/334] 10.222.0.105 () {56 vars in 1313 bytes} [Wed Aug 24 04:41:17 2022] GET /identity/v3/role_assignments?user.id=b62babf337c2497ebace81b4e375221c&scope.system=all&role.id=df27fa65390e4de78fa117773dc73555 => generated 512 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:18.925677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222231a6-defa-42c3-95d4-f42fba8970da None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:18.926185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222231a6-defa-42c3-95d4-f42fba8970da None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:18.926291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222231a6-defa-42c3-95d4-f42fba8970da None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:18.927923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 164/335] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:18.933755 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2fe70-586b-41e8-bc2f-2d60eacb11e2 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:18.934170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2fe70-586b-41e8-bc2f-2d60eacb11e2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:18.934347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2fe70-586b-41e8-bc2f-2d60eacb11e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:18.960040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-34f2fe70-586b-41e8-bc2f-2d60eacb11e2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:18.961254 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-34f2fe70-586b-41e8-bc2f-2d60eacb11e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:19.059689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 172/336] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:19.357637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7295f-2c4f-43e5-82fc-ba4c0fae810e None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:19.358037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7295f-2c4f-43e5-82fc-ba4c0fae810e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:19.358211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7295f-2c4f-43e5-82fc-ba4c0fae810e None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:19.359773 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 165/337] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:19.365823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4b915-8668-4402-81c7-f56e0a40af62 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:19.366115 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4b915-8668-4402-81c7-f56e0a40af62 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:19.366386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4b915-8668-4402-81c7-f56e0a40af62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:19.389343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d6c4b915-8668-4402-81c7-f56e0a40af62 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:19.390536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d6c4b915-8668-4402-81c7-f56e0a40af62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:19.477202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 173/338] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:19.614001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7295f-2c4f-43e5-82fc-ba4c0fae810e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:19.664697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e84d2-2674-4d14-82ae-3d4e224fbcab None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:19.665013 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e84d2-2674-4d14-82ae-3d4e224fbcab None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:19.665244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e84d2-2674-4d14-82ae-3d4e224fbcab None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:19.666581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 174/339] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:41:19.674407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b6048-88cf-47d3-a671-6c8f1bce6c29 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:19.674682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b6048-88cf-47d3-a671-6c8f1bce6c29 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:19.675504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b6048-88cf-47d3-a671-6c8f1bce6c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:19.696972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-0c4b6048-88cf-47d3-a671-6c8f1bce6c29 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:19.698150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0c4b6048-88cf-47d3-a671-6c8f1bce6c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:19.745781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:19.802460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 175/340] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:19 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:19.843721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7295f-2c4f-43e5-82fc-ba4c0fae810e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:19.868665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:19.886283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6b548-7027-43ee-9126-b24e31ce1d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:19.886942 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6b548-7027-43ee-9126-b24e31ce1d1d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:19.887247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6b548-7027-43ee-9126-b24e31ce1d1d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:19.887511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6b548-7027-43ee-9126-b24e31ce1d1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:19.904023 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6b548-7027-43ee-9126-b24e31ce1d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-94a6b548-7027-43ee-9126-b24e31ce1d1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:19.908840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:19.910536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:19.912121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:19.913567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 166/341] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:19.917960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b6048-88cf-47d3-a671-6c8f1bce6c29 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:19.924044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6b548-7027-43ee-9126-b24e31ce1d1d None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:20.084661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b6048-88cf-47d3-a671-6c8f1bce6c29 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:20.178858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2418bc-da4f-4da2-b558-5e1e392055eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:20.179503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2418bc-da4f-4da2-b558-5e1e392055eb None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:20.179780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2418bc-da4f-4da2-b558-5e1e392055eb None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:20.180321 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2418bc-da4f-4da2-b558-5e1e392055eb None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:20.181161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2418bc-da4f-4da2-b558-5e1e392055eb None admin] RBAC: Adding query filter params (name=member) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:20.181754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2418bc-da4f-4da2-b558-5e1e392055eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:20.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2418bc-da4f-4da2-b558-5e1e392055eb None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:20.194613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6b548-7027-43ee-9126-b24e31ce1d1d None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:20.195049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 176/342] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:19 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:20.200124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2418bc-da4f-4da2-b558-5e1e392055eb None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:20.247060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:20.264991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6819b0-ac77-4afc-9da4-1f58ffb49806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:20.265631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6819b0-ac77-4afc-9da4-1f58ffb49806 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:20.265908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6819b0-ac77-4afc-9da4-1f58ffb49806 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:20.266173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6819b0-ac77-4afc-9da4-1f58ffb49806 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:20.293100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6819b0-ac77-4afc-9da4-1f58ffb49806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-5a6819b0-ac77-4afc-9da4-1f58ffb49806 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:20.298119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:20.298999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:20.299873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:20.303999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 167/343] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:41:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:20.310047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6819b0-ac77-4afc-9da4-1f58ffb49806 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:20.345550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2418bc-da4f-4da2-b558-5e1e392055eb None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:20.366253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:20.400773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6448d-4f6b-4d0a-a095-c44f29ea6a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:20.401401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6448d-4f6b-4d0a-a095-c44f29ea6a0a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:20.401676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6448d-4f6b-4d0a-a095-c44f29ea6a0a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:20.401948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6448d-4f6b-4d0a-a095-c44f29ea6a0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:20.483475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6448d-4f6b-4d0a-a095-c44f29ea6a0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:20.485202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6448d-4f6b-4d0a-a095-c44f29ea6a0a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:20.485585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6819b0-ac77-4afc-9da4-1f58ffb49806 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:20.520175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:20.534418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0f44f-4a28-4576-bf0e-77c2d3770f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:20.535996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0f44f-4a28-4576-bf0e-77c2d3770f4b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:20.537786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0f44f-4a28-4576-bf0e-77c2d3770f4b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:20.537786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0f44f-4a28-4576-bf0e-77c2d3770f4b None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:20.537786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0f44f-4a28-4576-bf0e-77c2d3770f4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:20.537786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0f44f-4a28-4576-bf0e-77c2d3770f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:20.539589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0f44f-4a28-4576-bf0e-77c2d3770f4b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:20.551690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 168/344] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:20 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:20.557819 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dc1d5-f303-4b36-9775-3f54f7db7f30 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:20.558390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dc1d5-f303-4b36-9775-3f54f7db7f30 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:20.558861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678dc1d5-f303-4b36-9775-3f54f7db7f30 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:20.560703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 169/345] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:20 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:20.561615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678dc1d5-f303-4b36-9775-3f54f7db7f30 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:20.610191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 177/346] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:20 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:20.611390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6448d-4f6b-4d0a-a095-c44f29ea6a0a None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:20.679412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:20.684362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678dc1d5-f303-4b36-9775-3f54f7db7f30 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:20.724985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3eb59e5-903d-449f-bc58-b98f66da57c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:20.725792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb59e5-903d-449f-bc58-b98f66da57c9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:20.726166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb59e5-903d-449f-bc58-b98f66da57c9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:20.726640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb59e5-903d-449f-bc58-b98f66da57c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:20.737163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6448d-4f6b-4d0a-a095-c44f29ea6a0a None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:20.739508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3eb59e5-903d-449f-bc58-b98f66da57c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-d3eb59e5-903d-449f-bc58-b98f66da57c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:20.744770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:20.745775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 04:41:20.746632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:20.747263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 170/347] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:20.748732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcdd81-6cf7-487d-817a-da671171af0b None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:20.749018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcdd81-6cf7-487d-817a-da671171af0b None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:20.749288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcdd81-6cf7-487d-817a-da671171af0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:20.769676 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b2bcdd81-6cf7-487d-817a-da671171af0b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:20.771079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b2bcdd81-6cf7-487d-817a-da671171af0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:20.780276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:20.796950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31132f7-4fef-4a11-a4f6-62bbcd8b2892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:20.797623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31132f7-4fef-4a11-a4f6-62bbcd8b2892 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:20.797895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31132f7-4fef-4a11-a4f6-62bbcd8b2892 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:20.798160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31132f7-4fef-4a11-a4f6-62bbcd8b2892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:20.869908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 171/348] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:20.870951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcdd81-6cf7-487d-817a-da671171af0b None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:20.886198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31132f7-4fef-4a11-a4f6-62bbcd8b2892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:20.887697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31132f7-4fef-4a11-a4f6-62bbcd8b2892 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:20.961478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:20.993187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcdd81-6cf7-487d-817a-da671171af0b None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:21.020816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 178/349] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:20 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:21.021837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31132f7-4fef-4a11-a4f6-62bbcd8b2892 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:21.034667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7c808-ab00-42d4-9a74-707dd5451b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:21.035420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7c808-ab00-42d4-9a74-707dd5451b80 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.035700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7c808-ab00-42d4-9a74-707dd5451b80 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.035963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7c808-ab00-42d4-9a74-707dd5451b80 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.037546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7c808-ab00-42d4-9a74-707dd5451b80 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:21.038980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba7c808-ab00-42d4-9a74-707dd5451b80 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:21.040851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 172/350] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:20 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:21.041403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7c808-ab00-42d4-9a74-707dd5451b80 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:21.058000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:21.181512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7c808-ab00-42d4-9a74-707dd5451b80 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:21.190257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31132f7-4fef-4a11-a4f6-62bbcd8b2892 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:21.230556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27688e0c-2f1f-4f30-ba0d-eeb97740b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:21.231493 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75d20e-0c58-46c0-a3ae-9ea4697926a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:21.231976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27688e0c-2f1f-4f30-ba0d-eeb97740b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.231976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27688e0c-2f1f-4f30-ba0d-eeb97740b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.232148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27688e0c-2f1f-4f30-ba0d-eeb97740b4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.232690 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27688e0c-2f1f-4f30-ba0d-eeb97740b4c5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:21.232992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27688e0c-2f1f-4f30-ba0d-eeb97740b4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:21.233222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75d20e-0c58-46c0-a3ae-9ea4697926a3 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.233636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75d20e-0c58-46c0-a3ae-9ea4697926a3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.234044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a75d20e-0c58-46c0-a3ae-9ea4697926a3 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.234353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27688e0c-2f1f-4f30-ba0d-eeb97740b4c5 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:21.235518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75d20e-0c58-46c0-a3ae-9ea4697926a3 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:21.237775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a75d20e-0c58-46c0-a3ae-9ea4697926a3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:21.240953 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 173/351] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:21 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:21.241948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75d20e-0c58-46c0-a3ae-9ea4697926a3 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:21.244441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 179/352] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:21 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:21.246254 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729802f5-02dd-4b76-ab6c-746a51bcca28 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.246758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729802f5-02dd-4b76-ab6c-746a51bcca28 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.247285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729802f5-02dd-4b76-ab6c-746a51bcca28 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.248957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 180/353] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:21.249921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729802f5-02dd-4b76-ab6c-746a51bcca28 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:21.261081 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:21.365032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a75d20e-0c58-46c0-a3ae-9ea4697926a3 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:21.376079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:21.387785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729802f5-02dd-4b76-ab6c-746a51bcca28 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:21.417178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c5fa3-1c6a-4dc3-b6ff-c6b842ec0dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:21.418145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c5fa3-1c6a-4dc3-b6ff-c6b842ec0dd1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.418430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c5fa3-1c6a-4dc3-b6ff-c6b842ec0dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.418846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c5fa3-1c6a-4dc3-b6ff-c6b842ec0dd1 None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.425255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed491568-0dab-4a40-98b7-120454c3d924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:21.426720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed491568-0dab-4a40-98b7-120454c3d924 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.426995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed491568-0dab-4a40-98b7-120454c3d924 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.427305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed491568-0dab-4a40-98b7-120454c3d924 None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.428678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c5fa3-1c6a-4dc3-b6ff-c6b842ec0dd1 None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:21.428797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed491568-0dab-4a40-98b7-120454c3d924 None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:21.430674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed491568-0dab-4a40-98b7-120454c3d924 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:21.430781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c5fa3-1c6a-4dc3-b6ff-c6b842ec0dd1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:21.431794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 181/354] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:21 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:21.431948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 174/355] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:21 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:21.433086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed491568-0dab-4a40-98b7-120454c3d924 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:21.434433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcaf25-9652-45f5-b0f3-4a5d774aeb2a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.434718 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcaf25-9652-45f5-b0f3-4a5d774aeb2a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.434988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcaf25-9652-45f5-b0f3-4a5d774aeb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.457809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5adcaf25-9652-45f5-b0f3-4a5d774aeb2a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:21.459267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5adcaf25-9652-45f5-b0f3-4a5d774aeb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:21.543286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 175/356] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:21.543286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcaf25-9652-45f5-b0f3-4a5d774aeb2a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:21.551571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:21.563539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed491568-0dab-4a40-98b7-120454c3d924 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:21.606253 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbb79ee-eff7-46b2-bcc4-be6b378c626d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:21.606886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb79ee-eff7-46b2-bcc4-be6b378c626d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.607222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb79ee-eff7-46b2-bcc4-be6b378c626d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.607494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbb79ee-eff7-46b2-bcc4-be6b378c626d None admin] PATH_INFO: `/v3/users/8b2928fe42994d2d92ab0d7b2d23bda0` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.608642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb79ee-eff7-46b2-bcc4-be6b378c626d None admin] RBAC: Authorizing `identity:get_user(user_id=8b2928fe42994d2d92ab0d7b2d23bda0)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:21.610064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbb79ee-eff7-46b2-bcc4-be6b378c626d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:21.611827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 182/357] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:41:21 2022] GET /identity/v3/users/8b2928fe42994d2d92ab0d7b2d23bda0 => generated 294 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:21.617608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b1aad-fd0b-4361-bac7-4e56c36b06cd None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.617608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b1aad-fd0b-4361-bac7-4e56c36b06cd None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.617608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b1aad-fd0b-4361-bac7-4e56c36b06cd None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.617608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 183/358] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:21 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:21.617608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b1aad-fd0b-4361-bac7-4e56c36b06cd None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:21.648077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:21.677556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcaf25-9652-45f5-b0f3-4a5d774aeb2a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:21.723336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d5c87-af21-4c5f-a029-1b828b5a85b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:21.725280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d5c87-af21-4c5f-a029-1b828b5a85b9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.725563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d5c87-af21-4c5f-a029-1b828b5a85b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.725835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d5c87-af21-4c5f-a029-1b828b5a85b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.739483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d5c87-af21-4c5f-a029-1b828b5a85b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-071d5c87-af21-4c5f-a029-1b828b5a85b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:21.744264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:21.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:21.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 176/359] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:21.746638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d5c87-af21-4c5f-a029-1b828b5a85b9 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:21.757315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47b1aad-fd0b-4361-bac7-4e56c36b06cd None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:21.801835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21010354-153e-468d-8c7e-eb76d0a1e2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:21.802450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21010354-153e-468d-8c7e-eb76d0a1e2c4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.802729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21010354-153e-468d-8c7e-eb76d0a1e2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.803002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21010354-153e-468d-8c7e-eb76d0a1e2c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.803774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21010354-153e-468d-8c7e-eb76d0a1e2c4 None admin] RBAC: Adding query filter params (group.id=e9def1e9be924035925f06e5bff4df8b, role.id=b7f17471d2c948efa07d02424da68bea, scope.project.id=a1f41b226cc94cccbb39608d62cb6737) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:21.804079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21010354-153e-468d-8c7e-eb76d0a1e2c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:21.805521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21010354-153e-468d-8c7e-eb76d0a1e2c4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:21.811683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 184/360] 10.222.0.105 () {56 vars in 1381 bytes} [Wed Aug 24 04:41:21 2022] GET /identity/v3/role_assignments?group.id=e9def1e9be924035925f06e5bff4df8b&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=b7f17471d2c948efa07d02424da68bea => generated 261 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:21.813303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21010354-153e-468d-8c7e-eb76d0a1e2c4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:21.835396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:21.897148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d5c87-af21-4c5f-a029-1b828b5a85b9 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:21.943188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:21.943658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58658434-790c-4edf-b16c-602b6b21f901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:21.946489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58658434-790c-4edf-b16c-602b6b21f901 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.946489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58658434-790c-4edf-b16c-602b6b21f901 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.946489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58658434-790c-4edf-b16c-602b6b21f901 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.946489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58658434-790c-4edf-b16c-602b6b21f901 None admin] RBAC: Adding query filter params (group.id=e9def1e9be924035925f06e5bff4df8b, role.id=df27fa65390e4de78fa117773dc73555, scope.project.id=a1f41b226cc94cccbb39608d62cb6737) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:21.946489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58658434-790c-4edf-b16c-602b6b21f901 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:21.947352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58658434-790c-4edf-b16c-602b6b21f901 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:21.958040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 177/361] 10.222.0.105 () {56 vars in 1381 bytes} [Wed Aug 24 04:41:21 2022] GET /identity/v3/role_assignments?group.id=e9def1e9be924035925f06e5bff4df8b&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=df27fa65390e4de78fa117773dc73555 => generated 261 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:21.959718 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03552583-79dd-4079-a74d-d737a13475bf None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:21.960002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03552583-79dd-4079-a74d-d737a13475bf None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:21.960274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03552583-79dd-4079-a74d-d737a13475bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:21.962726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21010354-153e-468d-8c7e-eb76d0a1e2c4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:21.986272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-03552583-79dd-4079-a74d-d737a13475bf None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:21.988043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-03552583-79dd-4079-a74d-d737a13475bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:22.007969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bf37b-d485-4206-b982-b170f61acf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:22.008596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bf37b-d485-4206-b982-b170f61acf83 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:22.008876 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bf37b-d485-4206-b982-b170f61acf83 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:22.009146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bf37b-d485-4206-b982-b170f61acf83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:22.010024 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bf37b-d485-4206-b982-b170f61acf83 None admin] RBAC: Adding query filter params (role.id=0f01d78323f24797b5b1f2c555ea4c4c, user.id=8b2928fe42994d2d92ab0d7b2d23bda0, scope.system=all) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:22.010335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bf37b-d485-4206-b982-b170f61acf83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:22.011889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bf37b-d485-4206-b982-b170f61acf83 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:22.028908 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 185/362] 10.222.0.105 () {56 vars in 1313 bytes} [Wed Aug 24 04:41:21 2022] GET /identity/v3/role_assignments?user.id=8b2928fe42994d2d92ab0d7b2d23bda0&scope.system=all&role.id=0f01d78323f24797b5b1f2c555ea4c4c => generated 512 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:22.029790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bf37b-d485-4206-b982-b170f61acf83 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:22.113090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 178/363] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:22.114021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03552583-79dd-4079-a74d-d737a13475bf None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:22.201139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bf37b-d485-4206-b982-b170f61acf83 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:22.215385 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:22.265027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769ce81-f0e8-4b05-ac3d-d7b1be50fe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:22.265889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769ce81-f0e8-4b05-ac3d-d7b1be50fe20 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:22.266299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769ce81-f0e8-4b05-ac3d-d7b1be50fe20 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:22.266698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769ce81-f0e8-4b05-ac3d-d7b1be50fe20 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:22.267572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769ce81-f0e8-4b05-ac3d-d7b1be50fe20 None admin] RBAC: Adding query filter params (name=member) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:22.268025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769ce81-f0e8-4b05-ac3d-d7b1be50fe20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:22.270458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769ce81-f0e8-4b05-ac3d-d7b1be50fe20 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:22.274961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03552583-79dd-4079-a74d-d737a13475bf None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:22.281549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 186/364] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:22 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:22.288171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769ce81-f0e8-4b05-ac3d-d7b1be50fe20 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:22.307930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:22.350806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bce1ba-e5b6-433c-bed8-be9cba1ce6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:22.351484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bce1ba-e5b6-433c-bed8-be9cba1ce6e1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:22.351763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bce1ba-e5b6-433c-bed8-be9cba1ce6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:22.352036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bce1ba-e5b6-433c-bed8-be9cba1ce6e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:22.373519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bce1ba-e5b6-433c-bed8-be9cba1ce6e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-b0bce1ba-e5b6-433c-bed8-be9cba1ce6e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:22.384477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:22.385434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:22.386247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:22.390084 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 179/365] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:41:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:22.394153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bce1ba-e5b6-433c-bed8-be9cba1ce6e1 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:22.476305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769ce81-f0e8-4b05-ac3d-d7b1be50fe20 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:22.554690 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd30628-867b-41d9-a153-3b15e2cbb998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:22.555342 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd30628-867b-41d9-a153-3b15e2cbb998 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:22.555618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd30628-867b-41d9-a153-3b15e2cbb998 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:22.555887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd30628-867b-41d9-a153-3b15e2cbb998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:22.579676 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bce1ba-e5b6-433c-bed8-be9cba1ce6e1 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:22.603169 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:22.629059 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590aaf0-9f71-482b-b267-38c5a1ae51ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:22.630368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590aaf0-9f71-482b-b267-38c5a1ae51ab None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:22.630750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590aaf0-9f71-482b-b267-38c5a1ae51ab None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:22.631048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590aaf0-9f71-482b-b267-38c5a1ae51ab None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:22.631736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590aaf0-9f71-482b-b267-38c5a1ae51ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:22.632035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590aaf0-9f71-482b-b267-38c5a1ae51ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:22.633421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590aaf0-9f71-482b-b267-38c5a1ae51ab None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:22.643856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd30628-867b-41d9-a153-3b15e2cbb998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:22.645911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd30628-867b-41d9-a153-3b15e2cbb998 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:22.651734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 180/366] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:22 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:22.652818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590aaf0-9f71-482b-b267-38c5a1ae51ab None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:22.665732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:22.779251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 187/367] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:22 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:22.780773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd30628-867b-41d9-a153-3b15e2cbb998 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:22.791860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590aaf0-9f71-482b-b267-38c5a1ae51ab None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:22.830119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705e52c-a284-405c-9c6d-26bf9b9edb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:22.830300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705e52c-a284-405c-9c6d-26bf9b9edb36 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:22.830517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705e52c-a284-405c-9c6d-26bf9b9edb36 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:22.830723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705e52c-a284-405c-9c6d-26bf9b9edb36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:22.843071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705e52c-a284-405c-9c6d-26bf9b9edb36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-6705e52c-a284-405c-9c6d-26bf9b9edb36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:22.850029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:22.850955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:22.851815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 181/368] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:41:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:22.852486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705e52c-a284-405c-9c6d-26bf9b9edb36 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:22.910501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:22.912302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd30628-867b-41d9-a153-3b15e2cbb998 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:22.930510 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:22.959234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8c567-edf4-4cca-a27c-736fa12e3cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:22.960532 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8c567-edf4-4cca-a27c-736fa12e3cfd None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:22.960978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8c567-edf4-4cca-a27c-736fa12e3cfd None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:22.961597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8c567-edf4-4cca-a27c-736fa12e3cfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:22.980801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705e52c-a284-405c-9c6d-26bf9b9edb36 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:23.030951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc764f3-beea-4a4c-a6df-133d579b9a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:23.031629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc764f3-beea-4a4c-a6df-133d579b9a57 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:23.031907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc764f3-beea-4a4c-a6df-133d579b9a57 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:23.032279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc764f3-beea-4a4c-a6df-133d579b9a57 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:23.033416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc764f3-beea-4a4c-a6df-133d579b9a57 None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:23.034858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc764f3-beea-4a4c-a6df-133d579b9a57 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:23.038480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 182/369] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:22 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:23.039453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc764f3-beea-4a4c-a6df-133d579b9a57 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:23.051843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8c567-edf4-4cca-a27c-736fa12e3cfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:23.053298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8c567-edf4-4cca-a27c-736fa12e3cfd None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:23.156632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc764f3-beea-4a4c-a6df-133d579b9a57 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:23.161424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 188/370] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:22 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:23.162355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8c567-edf4-4cca-a27c-736fa12e3cfd None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:23.168363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:23.180004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:23.210191 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769b254-eb2d-41ac-acb2-9381ba3e04a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:23.212283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769b254-eb2d-41ac-acb2-9381ba3e04a8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:23.212717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769b254-eb2d-41ac-acb2-9381ba3e04a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:23.213157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769b254-eb2d-41ac-acb2-9381ba3e04a8 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:23.213965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769b254-eb2d-41ac-acb2-9381ba3e04a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:23.214410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769b254-eb2d-41ac-acb2-9381ba3e04a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:23.215966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769b254-eb2d-41ac-acb2-9381ba3e04a8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:23.226060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 183/371] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:23 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:23.227435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769b254-eb2d-41ac-acb2-9381ba3e04a8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:23.282498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8c567-edf4-4cca-a27c-736fa12e3cfd None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:23.328093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c2ae1-094a-48a3-baff-36f989934a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:23.329222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c2ae1-094a-48a3-baff-36f989934a0d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:23.330022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c2ae1-094a-48a3-baff-36f989934a0d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:23.330464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c2ae1-094a-48a3-baff-36f989934a0d None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:23.331765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c2ae1-094a-48a3-baff-36f989934a0d None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:23.334009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c2ae1-094a-48a3-baff-36f989934a0d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:23.337135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 189/372] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:23 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:23.338426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c2ae1-094a-48a3-baff-36f989934a0d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:23.346228 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769b254-eb2d-41ac-acb2-9381ba3e04a8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:23.401356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1c1ea-45d5-402a-a6a5-bad506b8873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:23.402029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1c1ea-45d5-402a-a6a5-bad506b8873e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:23.402313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1c1ea-45d5-402a-a6a5-bad506b8873e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:23.402586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1c1ea-45d5-402a-a6a5-bad506b8873e None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:23.406726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1c1ea-45d5-402a-a6a5-bad506b8873e None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:23.408173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1c1ea-45d5-402a-a6a5-bad506b8873e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:23.410109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 184/373] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:23 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:23.410714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1c1ea-45d5-402a-a6a5-bad506b8873e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:23.453872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:23.467841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:23.480733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c2ae1-094a-48a3-baff-36f989934a0d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:23.531600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0a4c5-d92e-40da-a70e-a1b2e9471b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:23.532983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0a4c5-d92e-40da-a70e-a1b2e9471b59 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:23.533440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0a4c5-d92e-40da-a70e-a1b2e9471b59 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:23.534127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0a4c5-d92e-40da-a70e-a1b2e9471b59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:23.552674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1c1ea-45d5-402a-a6a5-bad506b8873e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:23.594903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a33606-1b52-48e5-97b7-784503fa7f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:23.595835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a33606-1b52-48e5-97b7-784503fa7f80 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:23.596289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a33606-1b52-48e5-97b7-784503fa7f80 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:23.597150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a33606-1b52-48e5-97b7-784503fa7f80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:23.598058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a33606-1b52-48e5-97b7-784503fa7f80 None admin] RBAC: Adding query filter params (group.id=e9def1e9be924035925f06e5bff4df8b, role.id=df27fa65390e4de78fa117773dc73555, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:23.598579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a33606-1b52-48e5-97b7-784503fa7f80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:23.601441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a33606-1b52-48e5-97b7-784503fa7f80 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:23.609366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 185/374] 10.222.0.105 () {56 vars in 1381 bytes} [Wed Aug 24 04:41:23 2022] GET /identity/v3/role_assignments?group.id=e9def1e9be924035925f06e5bff4df8b&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=df27fa65390e4de78fa117773dc73555 => generated 261 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:23.610514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a33606-1b52-48e5-97b7-784503fa7f80 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:23.621023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0a4c5-d92e-40da-a70e-a1b2e9471b59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:23.622631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0a4c5-d92e-40da-a70e-a1b2e9471b59 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:23.767784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:23.796550 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a33606-1b52-48e5-97b7-784503fa7f80 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:23.798122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:23.802789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 190/375] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:23 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:23.809168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0a4c5-d92e-40da-a70e-a1b2e9471b59 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:23.845069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f0507-d67d-4fa3-a166-1bc23174bedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:23.846305 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f0507-d67d-4fa3-a166-1bc23174bedd None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:23.847210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f0507-d67d-4fa3-a166-1bc23174bedd None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:23.847921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f0507-d67d-4fa3-a166-1bc23174bedd None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:23.849335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f0507-d67d-4fa3-a166-1bc23174bedd None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:23.852271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f0507-d67d-4fa3-a166-1bc23174bedd None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:23.854383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 186/376] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:23 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:23.859805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f0507-d67d-4fa3-a166-1bc23174bedd None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:23.954249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0a4c5-d92e-40da-a70e-a1b2e9471b59 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:24.005603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f0507-d67d-4fa3-a166-1bc23174bedd None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:24.008147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57bb04-2040-4fc6-abc2-d6f73f517c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:24.008744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57bb04-2040-4fc6-abc2-d6f73f517c46 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:24.009129 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57bb04-2040-4fc6-abc2-d6f73f517c46 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:24.010498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57bb04-2040-4fc6-abc2-d6f73f517c46 None admin] PATH_INFO: `/v3/projects/e02e910be5274b36b8f63c2185a111d4` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:24.011834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57bb04-2040-4fc6-abc2-d6f73f517c46 None admin] RBAC: Authorizing `identity:get_project(project_id=e02e910be5274b36b8f63c2185a111d4)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:24.013490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57bb04-2040-4fc6-abc2-d6f73f517c46 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:24.016949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 191/377] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:23 2022] GET /identity/v3/projects/e02e910be5274b36b8f63c2185a111d4 => generated 345 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:24.021490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57bb04-2040-4fc6-abc2-d6f73f517c46 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:24.068780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e550c2-773b-4037-bd95-a0ebdd8fde2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:24.069410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e550c2-773b-4037-bd95-a0ebdd8fde2a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:24.069691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e550c2-773b-4037-bd95-a0ebdd8fde2a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:24.069961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e550c2-773b-4037-bd95-a0ebdd8fde2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:24.070702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e550c2-773b-4037-bd95-a0ebdd8fde2a None admin] RBAC: Adding query filter params (group.id=e9def1e9be924035925f06e5bff4df8b, role.id=b7f17471d2c948efa07d02424da68bea, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:24.071008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e550c2-773b-4037-bd95-a0ebdd8fde2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:24.072564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e550c2-773b-4037-bd95-a0ebdd8fde2a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:24.080424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 187/378] 10.222.0.105 () {56 vars in 1381 bytes} [Wed Aug 24 04:41:23 2022] GET /identity/v3/role_assignments?group.id=e9def1e9be924035925f06e5bff4df8b&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=b7f17471d2c948efa07d02424da68bea => generated 261 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:24.081248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:24.142865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57bb04-2040-4fc6-abc2-d6f73f517c46 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:24.188623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6eb92-7291-48e4-994b-128162d59d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:24.189511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6eb92-7291-48e4-994b-128162d59d13 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:24.189925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6eb92-7291-48e4-994b-128162d59d13 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:24.190327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6eb92-7291-48e4-994b-128162d59d13 None admin] PATH_INFO: `/v3/groups/275dcbbafdd94fb9ac32b82549bcdb48` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:24.201495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ae41d-ef76-4acb-910b-d30aa6152c37 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:24.201816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ae41d-ef76-4acb-910b-d30aa6152c37 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:24.202081 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ae41d-ef76-4acb-910b-d30aa6152c37 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:24.203781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 188/379] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:24.204211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6eb92-7291-48e4-994b-128162d59d13 None admin] RBAC: Authorizing `identity:get_group(group_id=275dcbbafdd94fb9ac32b82549bcdb48)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:24.205703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6eb92-7291-48e4-994b-128162d59d13 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:24.207431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 192/380] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:24 2022] GET /identity/v3/groups/275dcbbafdd94fb9ac32b82549bcdb48 => generated 230 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:24.214398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ae41d-ef76-4acb-910b-d30aa6152c37 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:24.214845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2aadf-6258-4c33-bbae-0580437a7e00 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:24.215160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2aadf-6258-4c33-bbae-0580437a7e00 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:24.215440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2aadf-6258-4c33-bbae-0580437a7e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:24.240394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-90c2aadf-6258-4c33-bbae-0580437a7e00 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:24.241911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-90c2aadf-6258-4c33-bbae-0580437a7e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:24.286974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:24.359490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ae41d-ef76-4acb-910b-d30aa6152c37 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:24.360363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 193/381] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:24.396831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacbbee9-20a1-4f78-b83a-98d1a83e9e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:24.397451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbbee9-20a1-4f78-b83a-98d1a83e9e8b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:24.397775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbbee9-20a1-4f78-b83a-98d1a83e9e8b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:24.398055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacbbee9-20a1-4f78-b83a-98d1a83e9e8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:24.398780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbbee9-20a1-4f78-b83a-98d1a83e9e8b None admin] RBAC: Adding query filter params (group.id=275dcbbafdd94fb9ac32b82549bcdb48, role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=e02e910be5274b36b8f63c2185a111d4) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:24.399199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbbee9-20a1-4f78-b83a-98d1a83e9e8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:24.400641 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacbbee9-20a1-4f78-b83a-98d1a83e9e8b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:24.406544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 189/382] 10.222.0.105 () {56 vars in 1381 bytes} [Wed Aug 24 04:41:24 2022] GET /identity/v3/role_assignments?group.id=275dcbbafdd94fb9ac32b82549bcdb48&scope.project.id=e02e910be5274b36b8f63c2185a111d4&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 261 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:24.688548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab8229-aab5-40ce-afef-7af261adc0e3 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:24.688941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab8229-aab5-40ce-afef-7af261adc0e3 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:24.689132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab8229-aab5-40ce-afef-7af261adc0e3 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:24.690706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 194/383] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:24.698101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab19306-447f-4923-9737-10998e8a101a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:24.698651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab19306-447f-4923-9737-10998e8a101a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:24.699167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab19306-447f-4923-9737-10998e8a101a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:24.714240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab8229-aab5-40ce-afef-7af261adc0e3 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:24.727268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2ab19306-447f-4923-9737-10998e8a101a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:24.728900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2ab19306-447f-4923-9737-10998e8a101a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:24.788101 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:24.851850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 190/384] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:24.884762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ab8229-aab5-40ce-afef-7af261adc0e3 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:24.925663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d6a2b-03dc-44b3-8dd2-05c851e4873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:24.926675 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d6a2b-03dc-44b3-8dd2-05c851e4873b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:24.927225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d6a2b-03dc-44b3-8dd2-05c851e4873b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:24.927906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d6a2b-03dc-44b3-8dd2-05c851e4873b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:24.940965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020d6a2b-03dc-44b3-8dd2-05c851e4873b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-020d6a2b-03dc-44b3-8dd2-05c851e4873b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:24.946645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:24.947544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:24.948328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:24.950934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 195/385] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:41:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:24.956341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab19306-447f-4923-9737-10998e8a101a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:24.986789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:25.078430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab19306-447f-4923-9737-10998e8a101a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:25.121443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073b602-4dc0-4687-8815-b3d495304bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:25.122065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073b602-4dc0-4687-8815-b3d495304bf7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:25.122335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073b602-4dc0-4687-8815-b3d495304bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:25.122600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073b602-4dc0-4687-8815-b3d495304bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:25.123355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073b602-4dc0-4687-8815-b3d495304bf7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:25.123650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073b602-4dc0-4687-8815-b3d495304bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:25.125211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073b602-4dc0-4687-8815-b3d495304bf7 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:25.137963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 191/386] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:24 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:25.143048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d6a2b-03dc-44b3-8dd2-05c851e4873b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:25.287480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d6a2b-03dc-44b3-8dd2-05c851e4873b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:25.288949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:25.339573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd70a4-c2d3-42d8-8728-96c093e938b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:25.340212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd70a4-c2d3-42d8-8728-96c093e938b9 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:25.340496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd70a4-c2d3-42d8-8728-96c093e938b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:25.340765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd70a4-c2d3-42d8-8728-96c093e938b9 None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:25.347720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd70a4-c2d3-42d8-8728-96c093e938b9 None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:25.349189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd70a4-c2d3-42d8-8728-96c093e938b9 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:25.351550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 196/387] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:25 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:25.357068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073b602-4dc0-4687-8815-b3d495304bf7 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:25.414819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd70a4-c2d3-42d8-8728-96c093e938b9 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:25.551370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:25.570467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073b602-4dc0-4687-8815-b3d495304bf7 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:25.651044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd70a4-c2d3-42d8-8728-96c093e938b9 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:25.676075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b05d-c756-4b1e-a1d6-4951bbf7e207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:25.676724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b05d-c756-4b1e-a1d6-4951bbf7e207 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:25.677002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b05d-c756-4b1e-a1d6-4951bbf7e207 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:25.677275 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b05d-c756-4b1e-a1d6-4951bbf7e207 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:25.679131 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11b05d-c756-4b1e-a1d6-4951bbf7e207 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:25.681501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11b05d-c756-4b1e-a1d6-4951bbf7e207 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:25.685152 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 192/388] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:25 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:25.691952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b05d-c756-4b1e-a1d6-4951bbf7e207 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:25.761281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecaa709-8535-4a74-899e-7b1146a269e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:25.761950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecaa709-8535-4a74-899e-7b1146a269e2 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:25.764806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecaa709-8535-4a74-899e-7b1146a269e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:25.764806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecaa709-8535-4a74-899e-7b1146a269e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:25.788909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecaa709-8535-4a74-899e-7b1146a269e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-6ecaa709-8535-4a74-899e-7b1146a269e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:25.794147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:25.795134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:25.796837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 197/389] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:25.804988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecaa709-8535-4a74-899e-7b1146a269e2 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:25.916280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b05d-c756-4b1e-a1d6-4951bbf7e207 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:25.920980 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:26.043785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38f695-d721-44ed-9f95-e44295d6c6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:26.044486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38f695-d721-44ed-9f95-e44295d6c6c3 None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:26.044769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38f695-d721-44ed-9f95-e44295d6c6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:26.045044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38f695-d721-44ed-9f95-e44295d6c6c3 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737/groups/e9def1e9be924035925f06e5bff4df8b/roles/b7f17471d2c948efa07d02424da68bea` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:26.056597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38f695-d721-44ed-9f95-e44295d6c6c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1f41b226cc94cccbb39608d62cb6737, group_id=e9def1e9be924035925f06e5bff4df8b, role_id=b7f17471d2c948efa07d02424da68bea)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:26.057982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38f695-d721-44ed-9f95-e44295d6c6c3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:26.147379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 193/390] 10.222.0.105 () {58 vars in 1447 bytes} [Wed Aug 24 04:41:25 2022] PUT /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737/groups/e9def1e9be924035925f06e5bff4df8b/roles/b7f17471d2c948efa07d02424da68bea => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:26.156776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecaa709-8535-4a74-899e-7b1146a269e2 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:26.226198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af722198-8ffc-4d1d-8929-46a0f0ffc45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:26.226838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af722198-8ffc-4d1d-8929-46a0f0ffc45a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:26.227191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af722198-8ffc-4d1d-8929-46a0f0ffc45a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:26.227802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af722198-8ffc-4d1d-8929-46a0f0ffc45a None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:26.228212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af722198-8ffc-4d1d-8929-46a0f0ffc45a None admin] RBAC: Adding query filter params (name=member) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:26.228518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af722198-8ffc-4d1d-8929-46a0f0ffc45a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:26.229966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af722198-8ffc-4d1d-8929-46a0f0ffc45a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:26.253711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 198/391] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:25 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 450 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:26.259685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38f695-d721-44ed-9f95-e44295d6c6c3 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:26.319526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:26.433901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38f695-d721-44ed-9f95-e44295d6c6c3 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:26.481380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92405029-0cfa-4987-a54c-b088b1598f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:26.482038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92405029-0cfa-4987-a54c-b088b1598f89 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:26.482318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92405029-0cfa-4987-a54c-b088b1598f89 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:26.482587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92405029-0cfa-4987-a54c-b088b1598f89 None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:26.484586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92405029-0cfa-4987-a54c-b088b1598f89 None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:26.486043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92405029-0cfa-4987-a54c-b088b1598f89 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:26.487962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 194/392] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:26 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:26.493811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af722198-8ffc-4d1d-8929-46a0f0ffc45a None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:26.539544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d77a4-13f9-471a-8713-542ed0310d90 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:26.539891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d77a4-13f9-471a-8713-542ed0310d90 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:26.540601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d77a4-13f9-471a-8713-542ed0310d90 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:26.542246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 195/393] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:26 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:41:26.552839 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37283e-d739-4bd5-a531-79ae4c871154 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:26.553625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37283e-d739-4bd5-a531-79ae4c871154 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:26.553625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37283e-d739-4bd5-a531-79ae4c871154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:26.582184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-4a37283e-d739-4bd5-a531-79ae4c871154 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:26.582184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4a37283e-d739-4bd5-a531-79ae4c871154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:26.691478 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:26.692697 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 196/394] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:26.704227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af722198-8ffc-4d1d-8929-46a0f0ffc45a None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:26.746657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f6a47-c3e4-48bc-877b-33ec3a144056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:26.747316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f6a47-c3e4-48bc-877b-33ec3a144056 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:26.747591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f6a47-c3e4-48bc-877b-33ec3a144056 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:26.747862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f6a47-c3e4-48bc-877b-33ec3a144056 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:26.749007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f6a47-c3e4-48bc-877b-33ec3a144056 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:26.750440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f6a47-c3e4-48bc-877b-33ec3a144056 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:26.752093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 199/395] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:26 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:26.758134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37283e-d739-4bd5-a531-79ae4c871154 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:26.785427 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2358385-7cdb-43f7-a549-39e57fff67eb None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:26.785767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2358385-7cdb-43f7-a549-39e57fff67eb None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:26.785980 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2358385-7cdb-43f7-a549-39e57fff67eb None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:26.786575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:26.787360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 200/396] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:26.793832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e4437-80d0-4fa1-95ad-cf4e9bf09b1c None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:26.794117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e4437-80d0-4fa1-95ad-cf4e9bf09b1c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:26.794393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e4437-80d0-4fa1-95ad-cf4e9bf09b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:26.821572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b02e4437-80d0-4fa1-95ad-cf4e9bf09b1c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:26.822789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b02e4437-80d0-4fa1-95ad-cf4e9bf09b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:26.891278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37283e-d739-4bd5-a531-79ae4c871154 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:26.930266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba5a7e-1dff-4164-8fdf-f30d0a7c9e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:26.931352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba5a7e-1dff-4164-8fdf-f30d0a7c9e9b None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:26.931845 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba5a7e-1dff-4164-8fdf-f30d0a7c9e9b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:26.932312 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba5a7e-1dff-4164-8fdf-f30d0a7c9e9b None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737/groups/e9def1e9be924035925f06e5bff4df8b/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:26.934501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba5a7e-1dff-4164-8fdf-f30d0a7c9e9b None admin] RBAC: Authorizing `identity:create_grant(project_id=a1f41b226cc94cccbb39608d62cb6737, group_id=e9def1e9be924035925f06e5bff4df8b, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:26.936429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba5a7e-1dff-4164-8fdf-f30d0a7c9e9b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:26.937992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 201/397] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:27.031376 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 197/398] 10.222.0.105 () {58 vars in 1447 bytes} [Wed Aug 24 04:41:26 2022] PUT /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737/groups/e9def1e9be924035925f06e5bff4df8b/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:41:27.070800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e4437-80d0-4fa1-95ad-cf4e9bf09b1c None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:27.109336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08364569-bf65-4d33-814a-909524261376 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:27.109683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08364569-bf65-4d33-814a-909524261376 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:27.110819 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08364569-bf65-4d33-814a-909524261376 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:27.112484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 198/399] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:27.120767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e8716-5349-4326-8332-f9dcdc4a7dff None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:27.121036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e8716-5349-4326-8332-f9dcdc4a7dff None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:27.126083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e8716-5349-4326-8332-f9dcdc4a7dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:27.149128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-117e8716-5349-4326-8332-f9dcdc4a7dff None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:27.150686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-117e8716-5349-4326-8332-f9dcdc4a7dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:27.243857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 199/400] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:27 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:27.244736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:27.258544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e4437-80d0-4fa1-95ad-cf4e9bf09b1c None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:27.279758 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e8716-5349-4326-8332-f9dcdc4a7dff None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:27.290631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:27.294264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa9c3-1c0d-41b5-8448-53aabb637d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:27.294902 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa9c3-1c0d-41b5-8448-53aabb637d00 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:27.295207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa9c3-1c0d-41b5-8448-53aabb637d00 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:27.295476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512aa9c3-1c0d-41b5-8448-53aabb637d00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:27.307194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512aa9c3-1c0d-41b5-8448-53aabb637d00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-512aa9c3-1c0d-41b5-8448-53aabb637d00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:27.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:27.313148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:27.313927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 202/401] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:27.319723 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa9c3-1c0d-41b5-8448-53aabb637d00 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:27.396738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e8716-5349-4326-8332-f9dcdc4a7dff None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:27.424519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512aa9c3-1c0d-41b5-8448-53aabb637d00 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:27.461256 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf4234-245e-417b-bc28-bf603ae76103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:27.461862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf4234-245e-417b-bc28-bf603ae76103 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:27.462137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf4234-245e-417b-bc28-bf603ae76103 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:27.462404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf4234-245e-417b-bc28-bf603ae76103 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:27.463170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf4234-245e-417b-bc28-bf603ae76103 None admin] RBAC: Adding query filter params (name=member) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:27.463470 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf4234-245e-417b-bc28-bf603ae76103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:27.464830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf4234-245e-417b-bc28-bf603ae76103 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:27.473653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 203/402] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:27 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:27.479271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf4234-245e-417b-bc28-bf603ae76103 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:27.497778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:27.512636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a48f8-57fd-49eb-b8a7-97f2949688e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:27.513291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a48f8-57fd-49eb-b8a7-97f2949688e0 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:27.521709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a48f8-57fd-49eb-b8a7-97f2949688e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:27.521709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a48f8-57fd-49eb-b8a7-97f2949688e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:27.533087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a48f8-57fd-49eb-b8a7-97f2949688e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-8f1a48f8-57fd-49eb-b8a7-97f2949688e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:27.538538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:27.539492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:27.540299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:27.551365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 200/403] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:41:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:27.558428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a48f8-57fd-49eb-b8a7-97f2949688e0 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:27.587756 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf4234-245e-417b-bc28-bf603ae76103 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:27.625290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9976f4-eef6-46ec-a265-f9275e5eb292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:27.625926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9976f4-eef6-46ec-a265-f9275e5eb292 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:27.626299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9976f4-eef6-46ec-a265-f9275e5eb292 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:27.626572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9976f4-eef6-46ec-a265-f9275e5eb292 None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:27.627725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9976f4-eef6-46ec-a265-f9275e5eb292 None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:27.629253 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9976f4-eef6-46ec-a265-f9275e5eb292 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:27.631200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 204/404] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:27 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:27.632599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9976f4-eef6-46ec-a265-f9275e5eb292 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:27.654183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:27.698884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a48f8-57fd-49eb-b8a7-97f2949688e0 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:27.731165 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:27.737849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773791f1-2b9c-4787-9e2b-12bea406c7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:27.738536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773791f1-2b9c-4787-9e2b-12bea406c7ab None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:27.738883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773791f1-2b9c-4787-9e2b-12bea406c7ab None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:27.739253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773791f1-2b9c-4787-9e2b-12bea406c7ab None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:27.740121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773791f1-2b9c-4787-9e2b-12bea406c7ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:27.740495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773791f1-2b9c-4787-9e2b-12bea406c7ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:27.742907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773791f1-2b9c-4787-9e2b-12bea406c7ab None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:27.754120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9976f4-eef6-46ec-a265-f9275e5eb292 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:27.759106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 201/405] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:27 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:27.760167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773791f1-2b9c-4787-9e2b-12bea406c7ab None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:27.801701 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0975834-3fa7-47d4-9a02-0dabe19c5b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:27.802890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0975834-3fa7-47d4-9a02-0dabe19c5b5c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:27.803361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0975834-3fa7-47d4-9a02-0dabe19c5b5c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:27.804452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0975834-3fa7-47d4-9a02-0dabe19c5b5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:27.817891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0975834-3fa7-47d4-9a02-0dabe19c5b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b0975834-3fa7-47d4-9a02-0dabe19c5b5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:27.825282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:27.826203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:27.827782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 205/406] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:41:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:27.830977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0975834-3fa7-47d4-9a02-0dabe19c5b5c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:27.884225 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773791f1-2b9c-4787-9e2b-12bea406c7ab None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:27.912727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:27.921399 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2fa53-9a22-45b7-9ca5-801b0e75c1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:27.922141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2fa53-9a22-45b7-9ca5-801b0e75c1bb None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:27.922592 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2fa53-9a22-45b7-9ca5-801b0e75c1bb None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:27.939614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2fa53-9a22-45b7-9ca5-801b0e75c1bb None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:27.939614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2fa53-9a22-45b7-9ca5-801b0e75c1bb None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:27.939614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2fa53-9a22-45b7-9ca5-801b0e75c1bb None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:27.939614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 202/407] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:27 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:27.939614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2fa53-9a22-45b7-9ca5-801b0e75c1bb None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:27.944345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0975834-3fa7-47d4-9a02-0dabe19c5b5c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:27.982305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414df0a2-4fbb-4841-82c8-fe50d4fb0d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:27.983301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414df0a2-4fbb-4841-82c8-fe50d4fb0d8f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:27.983721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414df0a2-4fbb-4841-82c8-fe50d4fb0d8f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:27.984218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414df0a2-4fbb-4841-82c8-fe50d4fb0d8f None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:27.985588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414df0a2-4fbb-4841-82c8-fe50d4fb0d8f None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:27.987200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414df0a2-4fbb-4841-82c8-fe50d4fb0d8f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:27.989060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 206/408] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:27 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:27.994633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414df0a2-4fbb-4841-82c8-fe50d4fb0d8f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:28.006809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:28.039444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2fa53-9a22-45b7-9ca5-801b0e75c1bb None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:28.077214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586a298-e99d-4c52-a423-6888e3d250ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:28.078095 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a298-e99d-4c52-a423-6888e3d250ff None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:28.078528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a298-e99d-4c52-a423-6888e3d250ff None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:28.079059 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586a298-e99d-4c52-a423-6888e3d250ff None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:28.079920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586a298-e99d-4c52-a423-6888e3d250ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:28.080439 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586a298-e99d-4c52-a423-6888e3d250ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:28.082090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586a298-e99d-4c52-a423-6888e3d250ff None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:28.092456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 203/409] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:27 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:28.093671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586a298-e99d-4c52-a423-6888e3d250ff None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:28.107507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414df0a2-4fbb-4841-82c8-fe50d4fb0d8f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:28.145333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46493cf3-f0c1-4f7d-90ed-47cb6145981e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:28.146133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46493cf3-f0c1-4f7d-90ed-47cb6145981e None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:28.146520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46493cf3-f0c1-4f7d-90ed-47cb6145981e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:28.147010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46493cf3-f0c1-4f7d-90ed-47cb6145981e None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/groups/e9def1e9be924035925f06e5bff4df8b/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:28.149682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46493cf3-f0c1-4f7d-90ed-47cb6145981e None admin] RBAC: Authorizing `identity:create_grant(project_id=ff12bdfc9e9745779c7f414a2d09d14b, group_id=e9def1e9be924035925f06e5bff4df8b, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:28.151314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46493cf3-f0c1-4f7d-90ed-47cb6145981e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:28.162146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:28.210647 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586a298-e99d-4c52-a423-6888e3d250ff None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:28.244085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773cb09-8c62-4ee1-b375-09ec489a0c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:28.244640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773cb09-8c62-4ee1-b375-09ec489a0c99 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:28.244917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773cb09-8c62-4ee1-b375-09ec489a0c99 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:28.245220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773cb09-8c62-4ee1-b375-09ec489a0c99 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:28.247936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773cb09-8c62-4ee1-b375-09ec489a0c99 None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:28.250031 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773cb09-8c62-4ee1-b375-09ec489a0c99 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:28.252278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 204/410] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:28 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:28.257093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773cb09-8c62-4ee1-b375-09ec489a0c99 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:28.259473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 207/411] 10.222.0.105 () {58 vars in 1447 bytes} [Wed Aug 24 04:41:27 2022] PUT /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/groups/e9def1e9be924035925f06e5bff4df8b/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:28.273376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46493cf3-f0c1-4f7d-90ed-47cb6145981e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:28.390227 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773cb09-8c62-4ee1-b375-09ec489a0c99 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:28.420643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:28.433667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7824a43-8a98-478c-8a2e-edde2dd2a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:28.434325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7824a43-8a98-478c-8a2e-edde2dd2a85e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:28.434610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7824a43-8a98-478c-8a2e-edde2dd2a85e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:28.434889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7824a43-8a98-478c-8a2e-edde2dd2a85e None admin] PATH_INFO: `/v3/groups/275dcbbafdd94fb9ac32b82549bcdb48` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:28.436367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7824a43-8a98-478c-8a2e-edde2dd2a85e None admin] RBAC: Authorizing `identity:get_group(group_id=275dcbbafdd94fb9ac32b82549bcdb48)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:28.437853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7824a43-8a98-478c-8a2e-edde2dd2a85e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:28.439519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 205/412] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:28 2022] GET /identity/v3/groups/275dcbbafdd94fb9ac32b82549bcdb48 => generated 230 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:28.440721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924a7a5-bb92-4424-b591-1f1ef2ff5744 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:28.441006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924a7a5-bb92-4424-b591-1f1ef2ff5744 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:28.441286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924a7a5-bb92-4424-b591-1f1ef2ff5744 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:28.442651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 206/413] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:28 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:28.443133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:28.448203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b627-fd59-45d7-9643-41f4929ac809 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:28.448485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b627-fd59-45d7-9643-41f4929ac809 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:28.448759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b627-fd59-45d7-9643-41f4929ac809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:28.468764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46493cf3-f0c1-4f7d-90ed-47cb6145981e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:28.470483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2e74b627-fd59-45d7-9643-41f4929ac809 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:28.471751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2e74b627-fd59-45d7-9643-41f4929ac809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:28.508521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640da06a-9bee-4172-81bb-564373813fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:28.509859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640da06a-9bee-4172-81bb-564373813fdb None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:28.510725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640da06a-9bee-4172-81bb-564373813fdb None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:28.511000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640da06a-9bee-4172-81bb-564373813fdb None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/groups/e9def1e9be924035925f06e5bff4df8b/roles/b7f17471d2c948efa07d02424da68bea` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:28.513205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640da06a-9bee-4172-81bb-564373813fdb None admin] RBAC: Authorizing `identity:create_grant(project_id=ff12bdfc9e9745779c7f414a2d09d14b, group_id=e9def1e9be924035925f06e5bff4df8b, role_id=b7f17471d2c948efa07d02424da68bea)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:28.514625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640da06a-9bee-4172-81bb-564373813fdb None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:28.551907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 207/414] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:28.552762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74b627-fd59-45d7-9643-41f4929ac809 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:28.586713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 208/415] 10.222.0.105 () {58 vars in 1447 bytes} [Wed Aug 24 04:41:28 2022] PUT /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b/groups/e9def1e9be924035925f06e5bff4df8b/roles/b7f17471d2c948efa07d02424da68bea => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:41:28.676640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74b627-fd59-45d7-9643-41f4929ac809 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:28.711914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:28.724856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773304f-05f5-445c-b27d-28dc3db70d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:28.730028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773304f-05f5-445c-b27d-28dc3db70d4e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:28.730028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773304f-05f5-445c-b27d-28dc3db70d4e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:28.730028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3773304f-05f5-445c-b27d-28dc3db70d4e None admin] PATH_INFO: `/v3/projects/e02e910be5274b36b8f63c2185a111d4` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:28.730028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773304f-05f5-445c-b27d-28dc3db70d4e None admin] RBAC: Authorizing `identity:get_project(project_id=e02e910be5274b36b8f63c2185a111d4)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:28.730028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3773304f-05f5-445c-b27d-28dc3db70d4e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:28.731904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 208/416] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:28 2022] GET /identity/v3/projects/e02e910be5274b36b8f63c2185a111d4 => generated 345 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:28.737594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640da06a-9bee-4172-81bb-564373813fdb None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:28.854443 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640da06a-9bee-4172-81bb-564373813fdb None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:28.884615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:28.892954 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54628b6-4ebf-4944-97d8-862598e81aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:28.893625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54628b6-4ebf-4944-97d8-862598e81aa0 None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:28.893904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54628b6-4ebf-4944-97d8-862598e81aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:28.894173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54628b6-4ebf-4944-97d8-862598e81aa0 None admin] PATH_INFO: `/v3/projects/e02e910be5274b36b8f63c2185a111d4/groups/275dcbbafdd94fb9ac32b82549bcdb48/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:28.896407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54628b6-4ebf-4944-97d8-862598e81aa0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e02e910be5274b36b8f63c2185a111d4, group_id=275dcbbafdd94fb9ac32b82549bcdb48, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:28.901623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54628b6-4ebf-4944-97d8-862598e81aa0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:28.932212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773304f-05f5-445c-b27d-28dc3db70d4e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:28.987986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 209/417] 10.222.0.105 () {58 vars in 1447 bytes} [Wed Aug 24 04:41:28 2022] PUT /identity/v3/projects/e02e910be5274b36b8f63c2185a111d4/groups/275dcbbafdd94fb9ac32b82549bcdb48/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:29.058878 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3773304f-05f5-445c-b27d-28dc3db70d4e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:29.111892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69c1c0-04f1-4e21-aa68-e034f222e915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:29.112553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c1c0-04f1-4e21-aa68-e034f222e915 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:29.112832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c1c0-04f1-4e21-aa68-e034f222e915 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:29.113100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69c1c0-04f1-4e21-aa68-e034f222e915 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:29.128238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69c1c0-04f1-4e21-aa68-e034f222e915 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-7f69c1c0-04f1-4e21-aa68-e034f222e915 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:29.133335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:29.134282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:29.135125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 209/418] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:41:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:29.144218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54628b6-4ebf-4944-97d8-862598e81aa0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:29.272117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e73ff-c231-4369-b895-79d42b2224aa None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:29.272438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e73ff-c231-4369-b895-79d42b2224aa None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:29.272672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e73ff-c231-4369-b895-79d42b2224aa None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:29.274489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54628b6-4ebf-4944-97d8-862598e81aa0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:29.274679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 210/419] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:29.284427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c8f4d-1e3c-447d-ab8f-74430b4a7673 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:29.284709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c8f4d-1e3c-447d-ab8f-74430b4a7673 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:29.284987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c8f4d-1e3c-447d-ab8f-74430b4a7673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:29.303660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-931c8f4d-1e3c-447d-ab8f-74430b4a7673 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:29.305436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-931c8f4d-1e3c-447d-ab8f-74430b4a7673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:29.325457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f5f10-c1f6-44c4-91cf-26bea65f9ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:29.326080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f5f10-c1f6-44c4-91cf-26bea65f9ff0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:29.326349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f5f10-c1f6-44c4-91cf-26bea65f9ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:29.326612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f5f10-c1f6-44c4-91cf-26bea65f9ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:29.327357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f5f10-c1f6-44c4-91cf-26bea65f9ff0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:29.327655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f5f10-c1f6-44c4-91cf-26bea65f9ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:29.329004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f5f10-c1f6-44c4-91cf-26bea65f9ff0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:29.349629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 210/420] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:29 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:29.354668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f5f10-c1f6-44c4-91cf-26bea65f9ff0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:29.411459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:29.434130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 211/421] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:29.481958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:29.562873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f5f10-c1f6-44c4-91cf-26bea65f9ff0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:29.606414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff230b-c4cc-409b-a59c-8fa64ad18bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:29.607050 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff230b-c4cc-409b-a59c-8fa64ad18bb7 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:29.607325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff230b-c4cc-409b-a59c-8fa64ad18bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:29.607598 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff230b-c4cc-409b-a59c-8fa64ad18bb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:29.687463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff230b-c4cc-409b-a59c-8fa64ad18bb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:29.688839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff230b-c4cc-409b-a59c-8fa64ad18bb7 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:29.749903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:29.772821 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c8f4d-1e3c-447d-ab8f-74430b4a7673 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:29.824461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 211/422] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:29 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:29.830746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff230b-c4cc-409b-a59c-8fa64ad18bb7 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:29.919772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c8f4d-1e3c-447d-ab8f-74430b4a7673 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:29.960031 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b30531-1cf5-43dc-b6bc-7e4792e96824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:29.960588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b30531-1cf5-43dc-b6bc-7e4792e96824 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:29.960940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b30531-1cf5-43dc-b6bc-7e4792e96824 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:29.961401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b30531-1cf5-43dc-b6bc-7e4792e96824 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:29.972900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b30531-1cf5-43dc-b6bc-7e4792e96824 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-00b30531-1cf5-43dc-b6bc-7e4792e96824 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:29.979612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:29.981707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:29.983723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 212/423] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:29.988764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b30531-1cf5-43dc-b6bc-7e4792e96824 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:30.035139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff230b-c4cc-409b-a59c-8fa64ad18bb7 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:30.067986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:30.082711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ef03e-1401-4b0c-be9a-61b328767af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:30.083383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ef03e-1401-4b0c-be9a-61b328767af9 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:30.083667 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ef03e-1401-4b0c-be9a-61b328767af9 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:30.084004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ef03e-1401-4b0c-be9a-61b328767af9 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:30.087705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ef03e-1401-4b0c-be9a-61b328767af9 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:30.089168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ef03e-1401-4b0c-be9a-61b328767af9 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:30.090811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 212/424] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:29 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:30.096303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ef03e-1401-4b0c-be9a-61b328767af9 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:30.129850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b30531-1cf5-43dc-b6bc-7e4792e96824 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:30.132405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:30.188400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feef274-34cf-4acc-bae4-6f5b6a57af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:30.189017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef274-34cf-4acc-bae4-6f5b6a57af07 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:30.189292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef274-34cf-4acc-bae4-6f5b6a57af07 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:30.189557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feef274-34cf-4acc-bae4-6f5b6a57af07 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:30.190211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feef274-34cf-4acc-bae4-6f5b6a57af07 None admin] RBAC: Adding query filter params (name=member) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:30.190505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feef274-34cf-4acc-bae4-6f5b6a57af07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:30.191886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feef274-34cf-4acc-bae4-6f5b6a57af07 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:30.212554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 213/425] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:29 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:30.220122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feef274-34cf-4acc-bae4-6f5b6a57af07 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:30.259153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ef03e-1401-4b0c-be9a-61b328767af9 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:30.299038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72801c35-a3aa-4097-86ee-ec1d19c34a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:30.299676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72801c35-a3aa-4097-86ee-ec1d19c34a22 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:30.299955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72801c35-a3aa-4097-86ee-ec1d19c34a22 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:30.301645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72801c35-a3aa-4097-86ee-ec1d19c34a22 None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:30.303546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72801c35-a3aa-4097-86ee-ec1d19c34a22 None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:30.305599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72801c35-a3aa-4097-86ee-ec1d19c34a22 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:30.307422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 213/426] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:30 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:30.313236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72801c35-a3aa-4097-86ee-ec1d19c34a22 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:30.430890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feef274-34cf-4acc-bae4-6f5b6a57af07 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:30.445304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:30.451054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:30.464343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72801c35-a3aa-4097-86ee-ec1d19c34a22 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:30.506107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10614f-82cf-408f-83f8-7d7350e8995d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:30.506727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10614f-82cf-408f-83f8-7d7350e8995d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:30.507005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10614f-82cf-408f-83f8-7d7350e8995d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:30.507301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10614f-82cf-408f-83f8-7d7350e8995d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:30.508110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10614f-82cf-408f-83f8-7d7350e8995d None admin] RBAC: Adding query filter params (group.id=e9def1e9be924035925f06e5bff4df8b, role.id=b7f17471d2c948efa07d02424da68bea, scope.project.id=a1f41b226cc94cccbb39608d62cb6737) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:30.508420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10614f-82cf-408f-83f8-7d7350e8995d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:30.509886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10614f-82cf-408f-83f8-7d7350e8995d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:30.519567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 214/427] 10.222.0.105 () {56 vars in 1381 bytes} [Wed Aug 24 04:41:30 2022] GET /identity/v3/role_assignments?group.id=e9def1e9be924035925f06e5bff4df8b&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=b7f17471d2c948efa07d02424da68bea => generated 613 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:30.520221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267a89e-043c-4922-a680-89abf0f42162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:30.520853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267a89e-043c-4922-a680-89abf0f42162 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:30.521131 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267a89e-043c-4922-a680-89abf0f42162 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:30.521403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267a89e-043c-4922-a680-89abf0f42162 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:30.597290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267a89e-043c-4922-a680-89abf0f42162 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:30.598752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267a89e-043c-4922-a680-89abf0f42162 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:30.735102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 214/428] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:30 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:30.740510 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10614f-82cf-408f-83f8-7d7350e8995d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:30.760351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd229af-39b5-4e59-a041-51e69928b5a6 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:30.760636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd229af-39b5-4e59-a041-51e69928b5a6 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:30.760913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd229af-39b5-4e59-a041-51e69928b5a6 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:30.762368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 215/429] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:30.767835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cafa3a-cb25-4d5b-b88e-e384bba46eca None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:30.768115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cafa3a-cb25-4d5b-b88e-e384bba46eca None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:30.768388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cafa3a-cb25-4d5b-b88e-e384bba46eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:30.778051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:30.791719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-f9cafa3a-cb25-4d5b-b88e-e384bba46eca None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:30.794151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f9cafa3a-cb25-4d5b-b88e-e384bba46eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:30.879262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10614f-82cf-408f-83f8-7d7350e8995d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:30.882224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 216/430] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:30.887079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47db392-5377-4375-aa96-726871ac567e None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:30.887079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47db392-5377-4375-aa96-726871ac567e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:30.887079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47db392-5377-4375-aa96-726871ac567e None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:30.887079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 217/431] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:30 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:30.893919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c37d2f-2eb9-417b-9b48-b84306b9abf9 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:30.894199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c37d2f-2eb9-417b-9b48-b84306b9abf9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:30.894469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c37d2f-2eb9-417b-9b48-b84306b9abf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:30.919355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c8c37d2f-2eb9-417b-9b48-b84306b9abf9 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:30.922597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c8c37d2f-2eb9-417b-9b48-b84306b9abf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:30.927725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d0e8d-2cda-4045-8ceb-6a71b1851f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:30.928364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d0e8d-2cda-4045-8ceb-6a71b1851f10 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:30.928642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d0e8d-2cda-4045-8ceb-6a71b1851f10 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:30.928916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d0e8d-2cda-4045-8ceb-6a71b1851f10 None admin] PATH_INFO: `/v3/projects/a1f41b226cc94cccbb39608d62cb6737` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:30.931880 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d0e8d-2cda-4045-8ceb-6a71b1851f10 None admin] RBAC: Authorizing `identity:get_project(project_id=a1f41b226cc94cccbb39608d62cb6737)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:30.933341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d0e8d-2cda-4045-8ceb-6a71b1851f10 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:30.939848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 215/432] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:30 2022] GET /identity/v3/projects/a1f41b226cc94cccbb39608d62cb6737 => generated 299 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:30.940675 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d0e8d-2cda-4045-8ceb-6a71b1851f10 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:30.963427 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:31.010071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 218/433] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:31.048020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d0e8d-2cda-4045-8ceb-6a71b1851f10 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:31.084526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c1053-bc80-40a2-be4f-9e27266ce7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:31.085236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1053-bc80-40a2-be4f-9e27266ce7e1 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:31.085581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1053-bc80-40a2-be4f-9e27266ce7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:31.085921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c1053-bc80-40a2-be4f-9e27266ce7e1 None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:31.087339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c1053-bc80-40a2-be4f-9e27266ce7e1 None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:31.088863 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c1053-bc80-40a2-be4f-9e27266ce7e1 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:31.090547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 216/434] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:30 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:31.096687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c37d2f-2eb9-417b-9b48-b84306b9abf9 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:31.161787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:31.165431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c1053-bc80-40a2-be4f-9e27266ce7e1 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:31.205903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c37d2f-2eb9-417b-9b48-b84306b9abf9 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:31.243950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf797f-210d-4b13-81ba-9776ac5ace3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:31.244928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf797f-210d-4b13-81ba-9776ac5ace3a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:31.245400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf797f-210d-4b13-81ba-9776ac5ace3a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:31.245879 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf797f-210d-4b13-81ba-9776ac5ace3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:31.246799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf797f-210d-4b13-81ba-9776ac5ace3a None admin] RBAC: Adding query filter params (group.id=e9def1e9be924035925f06e5bff4df8b, role.id=df27fa65390e4de78fa117773dc73555, scope.project.id=a1f41b226cc94cccbb39608d62cb6737) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:31.247356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf797f-210d-4b13-81ba-9776ac5ace3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:31.248957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf797f-210d-4b13-81ba-9776ac5ace3a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:31.263274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 219/435] 10.222.0.105 () {56 vars in 1381 bytes} [Wed Aug 24 04:41:31 2022] GET /identity/v3/role_assignments?group.id=e9def1e9be924035925f06e5bff4df8b&scope.project.id=a1f41b226cc94cccbb39608d62cb6737&role.id=df27fa65390e4de78fa117773dc73555 => generated 613 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:31.282784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:31.300456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c1053-bc80-40a2-be4f-9e27266ce7e1 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:31.304182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf797f-210d-4b13-81ba-9776ac5ace3a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:31.356531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c59ad-75ac-4a85-a924-ebe961f691e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:31.357179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c59ad-75ac-4a85-a924-ebe961f691e2 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:31.357487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c59ad-75ac-4a85-a924-ebe961f691e2 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:31.357983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c59ad-75ac-4a85-a924-ebe961f691e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:31.374221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c59ad-75ac-4a85-a924-ebe961f691e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-825c59ad-75ac-4a85-a924-ebe961f691e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:31.380861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:31.381829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 04:41:31.382691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:31.388859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 217/436] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:31.390918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c59ad-75ac-4a85-a924-ebe961f691e2 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:31.417653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf797f-210d-4b13-81ba-9776ac5ace3a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:31.448145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:31.456728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd06869-1ed2-4a61-bd06-6b8f5a9280be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:31.457799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd06869-1ed2-4a61-bd06-6b8f5a9280be None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:31.457799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd06869-1ed2-4a61-bd06-6b8f5a9280be None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:31.458011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd06869-1ed2-4a61-bd06-6b8f5a9280be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:31.470297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd06869-1ed2-4a61-bd06-6b8f5a9280be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-1fd06869-1ed2-4a61-bd06-6b8f5a9280be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:31.475720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:31.476835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:31.478753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 220/437] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:41:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:31.482662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd06869-1ed2-4a61-bd06-6b8f5a9280be None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:31.534845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c59ad-75ac-4a85-a924-ebe961f691e2 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:31.580428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9119b8e-648a-4913-b4d1-a58670689ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:31.580428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9119b8e-648a-4913-b4d1-a58670689ef3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:31.580428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9119b8e-648a-4913-b4d1-a58670689ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:31.580428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9119b8e-648a-4913-b4d1-a58670689ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:31.580826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9119b8e-648a-4913-b4d1-a58670689ef3 None admin] RBAC: Adding query filter params (name=member) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:31.580826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9119b8e-648a-4913-b4d1-a58670689ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:31.582528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9119b8e-648a-4913-b4d1-a58670689ef3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:31.591835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 218/438] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:41:31 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:31.597432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9119b8e-648a-4913-b4d1-a58670689ef3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:31.613198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:31.627505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd06869-1ed2-4a61-bd06-6b8f5a9280be None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:31.669170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4570b8a-1c07-44e0-bd5a-6fe66bdcf1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:31.670157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4570b8a-1c07-44e0-bd5a-6fe66bdcf1b7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:31.670605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4570b8a-1c07-44e0-bd5a-6fe66bdcf1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:31.670876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4570b8a-1c07-44e0-bd5a-6fe66bdcf1b7 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:31.672700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4570b8a-1c07-44e0-bd5a-6fe66bdcf1b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:31.673012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4570b8a-1c07-44e0-bd5a-6fe66bdcf1b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:31.674489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4570b8a-1c07-44e0-bd5a-6fe66bdcf1b7 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:31.684744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 221/439] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:31 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:31.686055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d315d-a70f-4d16-900f-e5013f6eaad5 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:31.686347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d315d-a70f-4d16-900f-e5013f6eaad5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:31.687087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d315d-a70f-4d16-900f-e5013f6eaad5 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:31.688992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 222/440] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:31.691929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d315d-a70f-4d16-900f-e5013f6eaad5 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:31.722539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9119b8e-648a-4913-b4d1-a58670689ef3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:31.756573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:31.768420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19b0b2-0277-4f85-8382-5b5db79840e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:31.777239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b0b2-0277-4f85-8382-5b5db79840e4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:31.777239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b0b2-0277-4f85-8382-5b5db79840e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:31.777239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19b0b2-0277-4f85-8382-5b5db79840e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:31.822167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d315d-a70f-4d16-900f-e5013f6eaad5 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:31.851035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19b0b2-0277-4f85-8382-5b5db79840e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:31.854651 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19b0b2-0277-4f85-8382-5b5db79840e4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:31.863184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf254844-a00c-4f8f-950b-81cebe5fc158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:31.863772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:31.863902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf254844-a00c-4f8f-950b-81cebe5fc158 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:31.864075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf254844-a00c-4f8f-950b-81cebe5fc158 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:31.864322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf254844-a00c-4f8f-950b-81cebe5fc158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:31.930029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf254844-a00c-4f8f-950b-81cebe5fc158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:31.932155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf254844-a00c-4f8f-950b-81cebe5fc158 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:31.952278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 219/441] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 04:41:31.953691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f222c9a-4cdf-4ce1-ab10-4af908cefe23 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:31.954123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f222c9a-4cdf-4ce1-ab10-4af908cefe23 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:31.954567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f222c9a-4cdf-4ce1-ab10-4af908cefe23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:31.972191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-9f222c9a-4cdf-4ce1-ab10-4af908cefe23 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:31.973381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9f222c9a-4cdf-4ce1-ab10-4af908cefe23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:31.979979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:32.029634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 223/442] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:31 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:32.030586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf254844-a00c-4f8f-950b-81cebe5fc158 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:32.040433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 220/443] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:32.041082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f222c9a-4cdf-4ce1-ab10-4af908cefe23 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:32.087613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:32.135580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf254844-a00c-4f8f-950b-81cebe5fc158 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:32.151473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f222c9a-4cdf-4ce1-ab10-4af908cefe23 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:32.175593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213337f-8fef-49dc-932b-524828980008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:32.176220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213337f-8fef-49dc-932b-524828980008 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:32.176491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213337f-8fef-49dc-932b-524828980008 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:32.176754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213337f-8fef-49dc-932b-524828980008 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:32.177891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213337f-8fef-49dc-932b-524828980008 None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:32.179343 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213337f-8fef-49dc-932b-524828980008 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:32.182106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 224/444] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:32 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:32.188799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213337f-8fef-49dc-932b-524828980008 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:32.194630 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82846b6d-f608-41d9-93ff-da2b67c832f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:32.195305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82846b6d-f608-41d9-93ff-da2b67c832f7 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:32.195585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82846b6d-f608-41d9-93ff-da2b67c832f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:32.195853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82846b6d-f608-41d9-93ff-da2b67c832f7 None admin] PATH_INFO: `/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:32.197073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82846b6d-f608-41d9-93ff-da2b67c832f7 None admin] RBAC: Authorizing `identity:get_project(project_id=ff12bdfc9e9745779c7f414a2d09d14b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:32.198547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82846b6d-f608-41d9-93ff-da2b67c832f7 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:32.204729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 221/445] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:32 2022] GET /identity/v3/projects/ff12bdfc9e9745779c7f414a2d09d14b => generated 303 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:32.205692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82846b6d-f608-41d9-93ff-da2b67c832f7 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:32.213615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:32.298031 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213337f-8fef-49dc-932b-524828980008 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:32.306460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82846b6d-f608-41d9-93ff-da2b67c832f7 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:32.313987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:32.333136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da19b0-011c-44fd-9256-fe5cf429270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:32.333754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da19b0-011c-44fd-9256-fe5cf429270a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:32.334028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da19b0-011c-44fd-9256-fe5cf429270a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:32.334294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da19b0-011c-44fd-9256-fe5cf429270a None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:32.335488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da19b0-011c-44fd-9256-fe5cf429270a None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:32.336966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da19b0-011c-44fd-9256-fe5cf429270a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:32.338565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 225/446] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:32 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:32.339301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da19b0-011c-44fd-9256-fe5cf429270a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:32.345293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f734d31-8bfd-41a7-861c-f65f087e08c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:32.345933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f734d31-8bfd-41a7-861c-f65f087e08c4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:32.346216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f734d31-8bfd-41a7-861c-f65f087e08c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:32.346491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f734d31-8bfd-41a7-861c-f65f087e08c4 None admin] PATH_INFO: `/v3/groups/e9def1e9be924035925f06e5bff4df8b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:32.347719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f734d31-8bfd-41a7-861c-f65f087e08c4 None admin] RBAC: Authorizing `identity:get_group(group_id=e9def1e9be924035925f06e5bff4df8b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:32.349458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f734d31-8bfd-41a7-861c-f65f087e08c4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:32.350782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 222/447] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:32 2022] GET /identity/v3/groups/e9def1e9be924035925f06e5bff4df8b => generated 227 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:32.351476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f734d31-8bfd-41a7-861c-f65f087e08c4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:32.426346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:32.440953 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da19b0-011c-44fd-9256-fe5cf429270a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:32.451041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f734d31-8bfd-41a7-861c-f65f087e08c4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:32.474691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2413e-7ffd-4721-9c4d-906fd4d900a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:32.475340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2413e-7ffd-4721-9c4d-906fd4d900a2 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:32.475613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2413e-7ffd-4721-9c4d-906fd4d900a2 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:32.475880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2413e-7ffd-4721-9c4d-906fd4d900a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:32.485127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45024372-9063-42e8-8b39-232105632d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:32.485725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45024372-9063-42e8-8b39-232105632d0f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:32.486006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45024372-9063-42e8-8b39-232105632d0f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:32.486281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45024372-9063-42e8-8b39-232105632d0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:32.486372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2413e-7ffd-4721-9c4d-906fd4d900a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:32.487006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45024372-9063-42e8-8b39-232105632d0f None admin] RBAC: Adding query filter params (group.id=e9def1e9be924035925f06e5bff4df8b, role.id=df27fa65390e4de78fa117773dc73555, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:32.487346 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45024372-9063-42e8-8b39-232105632d0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:32.488759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45024372-9063-42e8-8b39-232105632d0f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-1ef2413e-7ffd-4721-9c4d-906fd4d900a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:32.490970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:32.491894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:32.492804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 226/448] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:41:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:32.493481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2413e-7ffd-4721-9c4d-906fd4d900a2 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:32.497503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 223/449] 10.222.0.105 () {56 vars in 1381 bytes} [Wed Aug 24 04:41:32 2022] GET /identity/v3/role_assignments?group.id=e9def1e9be924035925f06e5bff4df8b&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=df27fa65390e4de78fa117773dc73555 => generated 613 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:32.498350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45024372-9063-42e8-8b39-232105632d0f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:32.544101 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:32.615365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45024372-9063-42e8-8b39-232105632d0f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:32.616220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2413e-7ffd-4721-9c4d-906fd4d900a2 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:32.656744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b79b4-4ece-439b-af19-0bba5f18426f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:32.656967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e0b9b-bf6d-4f50-a724-ea48c10d6245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:32.657348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b79b4-4ece-439b-af19-0bba5f18426f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:32.657540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0b9b-bf6d-4f50-a724-ea48c10d6245 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:32.657638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b79b4-4ece-439b-af19-0bba5f18426f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:32.657820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0b9b-bf6d-4f50-a724-ea48c10d6245 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:32.657916 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b79b4-4ece-439b-af19-0bba5f18426f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:32.658091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0b9b-bf6d-4f50-a724-ea48c10d6245 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:32.658581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b79b4-4ece-439b-af19-0bba5f18426f None admin] RBAC: Adding query filter params (group.id=e9def1e9be924035925f06e5bff4df8b, role.id=b7f17471d2c948efa07d02424da68bea, scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:32.658752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e0b9b-bf6d-4f50-a724-ea48c10d6245 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:32.658930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b79b4-4ece-439b-af19-0bba5f18426f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:32.660328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b79b4-4ece-439b-af19-0bba5f18426f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:32.661148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e0b9b-bf6d-4f50-a724-ea48c10d6245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:32.662803 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e0b9b-bf6d-4f50-a724-ea48c10d6245 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:32.668230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 227/450] 10.222.0.105 () {56 vars in 1381 bytes} [Wed Aug 24 04:41:32 2022] GET /identity/v3/role_assignments?group.id=e9def1e9be924035925f06e5bff4df8b&scope.project.id=ff12bdfc9e9745779c7f414a2d09d14b&role.id=b7f17471d2c948efa07d02424da68bea => generated 613 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:32.675060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 224/451] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:32 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:32.678661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b79b4-4ece-439b-af19-0bba5f18426f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:32.693763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:32.803655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b79b4-4ece-439b-af19-0bba5f18426f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:32.841543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbdd0d-6cce-44b2-ba42-8bf6150ae401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:32.842178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbdd0d-6cce-44b2-ba42-8bf6150ae401 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:32.842426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbdd0d-6cce-44b2-ba42-8bf6150ae401 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:32.842670 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cbdd0d-6cce-44b2-ba42-8bf6150ae401 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:32.913259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbdd0d-6cce-44b2-ba42-8bf6150ae401 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:32.914633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cbdd0d-6cce-44b2-ba42-8bf6150ae401 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:32.926413 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:33.017069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 228/452] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:32 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:33.023045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e0b9b-bf6d-4f50-a724-ea48c10d6245 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:33.121403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:33.124398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e0b9b-bf6d-4f50-a724-ea48c10d6245 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:33.157092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5f217-5bcd-419a-b2cb-5c0e9fedbe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:33.157709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5f217-5bcd-419a-b2cb-5c0e9fedbe9b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:33.157987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5f217-5bcd-419a-b2cb-5c0e9fedbe9b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:33.158258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde5f217-5bcd-419a-b2cb-5c0e9fedbe9b None admin] PATH_INFO: `/v3/projects/e02e910be5274b36b8f63c2185a111d4` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:33.159401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5f217-5bcd-419a-b2cb-5c0e9fedbe9b None admin] RBAC: Authorizing `identity:get_project(project_id=e02e910be5274b36b8f63c2185a111d4)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:33.160841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde5f217-5bcd-419a-b2cb-5c0e9fedbe9b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:33.162621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 225/453] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:41:33 2022] GET /identity/v3/projects/e02e910be5274b36b8f63c2185a111d4 => generated 345 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:33.169306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbdd0d-6cce-44b2-ba42-8bf6150ae401 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:33.264410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cbdd0d-6cce-44b2-ba42-8bf6150ae401 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:33.274491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:33.297980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3a130-1f20-47e3-b2a1-238367f8cee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:33.298618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3a130-1f20-47e3-b2a1-238367f8cee4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:33.298937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3a130-1f20-47e3-b2a1-238367f8cee4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:33.299237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3a130-1f20-47e3-b2a1-238367f8cee4 None admin] PATH_INFO: `/v3/groups/275dcbbafdd94fb9ac32b82549bcdb48` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:33.300335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3a130-1f20-47e3-b2a1-238367f8cee4 None admin] RBAC: Authorizing `identity:get_group(group_id=275dcbbafdd94fb9ac32b82549bcdb48)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:33.301762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3a130-1f20-47e3-b2a1-238367f8cee4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:33.303341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 229/454] 10.222.0.105 () {56 vars in 1185 bytes} [Wed Aug 24 04:41:33 2022] GET /identity/v3/groups/275dcbbafdd94fb9ac32b82549bcdb48 => generated 230 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:33.309430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5f217-5bcd-419a-b2cb-5c0e9fedbe9b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:33.406046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde5f217-5bcd-419a-b2cb-5c0e9fedbe9b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:33.439080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d6f11-6eda-473f-a661-eb383ef0b230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:33.439634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d6f11-6eda-473f-a661-eb383ef0b230 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:33.439913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d6f11-6eda-473f-a661-eb383ef0b230 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:33.440185 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d6f11-6eda-473f-a661-eb383ef0b230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:33.440903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d6f11-6eda-473f-a661-eb383ef0b230 None admin] RBAC: Adding query filter params (group.id=275dcbbafdd94fb9ac32b82549bcdb48, role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=e02e910be5274b36b8f63c2185a111d4) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:33.441205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d6f11-6eda-473f-a661-eb383ef0b230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:33.442618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d6f11-6eda-473f-a661-eb383ef0b230 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:33.448333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 226/455] 10.222.0.105 () {56 vars in 1381 bytes} [Wed Aug 24 04:41:33 2022] GET /identity/v3/role_assignments?group.id=275dcbbafdd94fb9ac32b82549bcdb48&scope.project.id=e02e910be5274b36b8f63c2185a111d4&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 613 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:35.780628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4759d4-9ba0-47cb-8736-0b412ca641cc None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:35.781138 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4759d4-9ba0-47cb-8736-0b412ca641cc None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:35.781246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4759d4-9ba0-47cb-8736-0b412ca641cc None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:35.782730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 230/456] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:35.788318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7229703-82f4-4ce9-bac1-4048446c0ffe None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:35.789163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7229703-82f4-4ce9-bac1-4048446c0ffe None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:35.789657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7229703-82f4-4ce9-bac1-4048446c0ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:35.808541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f7229703-82f4-4ce9-bac1-4048446c0ffe None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:35.810065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f7229703-82f4-4ce9-bac1-4048446c0ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:35.837280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:35.895370 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 227/457] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:35.996192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c92935-edb5-4335-b27f-83b57a928431 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:35.996815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c92935-edb5-4335-b27f-83b57a928431 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:35.997335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c92935-edb5-4335-b27f-83b57a928431 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:35.998865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 231/458] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:35 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:36.012646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa46ea-a1b6-4d31-921f-b8949eca9337 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:36.013199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa46ea-a1b6-4d31-921f-b8949eca9337 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:36.013693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa46ea-a1b6-4d31-921f-b8949eca9337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:36.037319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-88fa46ea-a1b6-4d31-921f-b8949eca9337 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:36.038792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-88fa46ea-a1b6-4d31-921f-b8949eca9337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:36.066999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6baa-d6e7-4ec8-ae90-fba273f30420 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:36.068883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6baa-d6e7-4ec8-ae90-fba273f30420 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:36.068883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff6baa-d6e7-4ec8-ae90-fba273f30420 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:36.068883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 232/459] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:36.076094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ce49d-ac11-4f8d-9dfe-b9a978da28ca None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:36.076371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ce49d-ac11-4f8d-9dfe-b9a978da28ca None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:36.076637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ce49d-ac11-4f8d-9dfe-b9a978da28ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:36.109811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7c6ce49d-ac11-4f8d-9dfe-b9a978da28ca None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:36.111299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7c6ce49d-ac11-4f8d-9dfe-b9a978da28ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:36.158441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 228/460] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:36.220455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 233/461] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:36.258861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa46ea-a1b6-4d31-921f-b8949eca9337 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:36.359816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:36.369413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa46ea-a1b6-4d31-921f-b8949eca9337 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:36.405910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2998c7-d650-4368-bd53-2bf44d99b8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:36.406555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2998c7-d650-4368-bd53-2bf44d99b8b2 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:36.406842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2998c7-d650-4368-bd53-2bf44d99b8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:36.407150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2998c7-d650-4368-bd53-2bf44d99b8b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:36.418074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2998c7-d650-4368-bd53-2bf44d99b8b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:36.419624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2998c7-d650-4368-bd53-2bf44d99b8b2 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-ff2998c7-d650-4368-bd53-2bf44d99b8b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:36.434938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:36.436079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:36.437143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:36.437143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:36.437143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:36.439665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 229/462] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:36.445328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ce49d-ac11-4f8d-9dfe-b9a978da28ca None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:36.519097 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2998c7-d650-4368-bd53-2bf44d99b8b2 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:36.575876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ce49d-ac11-4f8d-9dfe-b9a978da28ca None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:36.603875 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:36.616108 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068bd87-b51d-448c-9715-98504296577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:36.616646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068bd87-b51d-448c-9715-98504296577c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:36.616918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068bd87-b51d-448c-9715-98504296577c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:36.617183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068bd87-b51d-448c-9715-98504296577c None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:36.617816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068bd87-b51d-448c-9715-98504296577c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:36.618110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068bd87-b51d-448c-9715-98504296577c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:36.619781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068bd87-b51d-448c-9715-98504296577c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:36.648648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 234/463] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:36.658303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068bd87-b51d-448c-9715-98504296577c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:36.692839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2998c7-d650-4368-bd53-2bf44d99b8b2 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:36.744178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671a904-4849-4fdd-9f7b-ec2c7bcb261c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:36.745139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671a904-4849-4fdd-9f7b-ec2c7bcb261c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:36.745341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671a904-4849-4fdd-9f7b-ec2c7bcb261c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:36.745533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671a904-4849-4fdd-9f7b-ec2c7bcb261c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:36.759781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671a904-4849-4fdd-9f7b-ec2c7bcb261c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:36.762496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671a904-4849-4fdd-9f7b-ec2c7bcb261c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:36.764129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068bd87-b51d-448c-9715-98504296577c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:36.765217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b671a904-4849-4fdd-9f7b-ec2c7bcb261c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:36.783211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:36.784277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:36.785118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:36.785118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:36.785118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:36.787102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 230/464] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:36.788293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671a904-4849-4fdd-9f7b-ec2c7bcb261c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:36.808783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74cf8b-8f0a-4b9e-86c1-dc1d87a1ea53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:36.810087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74cf8b-8f0a-4b9e-86c1-dc1d87a1ea53 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:36.810628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74cf8b-8f0a-4b9e-86c1-dc1d87a1ea53 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:36.811139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74cf8b-8f0a-4b9e-86c1-dc1d87a1ea53 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:36.812085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d74cf8b-8f0a-4b9e-86c1-dc1d87a1ea53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:36.813729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d74cf8b-8f0a-4b9e-86c1-dc1d87a1ea53 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:36.900798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671a904-4849-4fdd-9f7b-ec2c7bcb261c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:36.916219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 235/465] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:41:36 2022] POST /identity/v3/users => generated 247 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:41:36.918030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74cf8b-8f0a-4b9e-86c1-dc1d87a1ea53 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:36.946533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458b86a-3cda-4352-839e-2931f776ef08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:36.947689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458b86a-3cda-4352-839e-2931f776ef08 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:36.948019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458b86a-3cda-4352-839e-2931f776ef08 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:36.948373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458b86a-3cda-4352-839e-2931f776ef08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:36.953127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:36.962527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458b86a-3cda-4352-839e-2931f776ef08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:36.964847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458b86a-3cda-4352-839e-2931f776ef08 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-2458b86a-3cda-4352-839e-2931f776ef08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:36.981674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:36.982840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:36.983933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:36.983933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:36.983933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:36.985721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 231/466] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:36.987190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a639f-4dc0-4d96-8c6b-9492fdcc1769 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:36.987618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a639f-4dc0-4d96-8c6b-9492fdcc1769 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:36.988014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a639f-4dc0-4d96-8c6b-9492fdcc1769 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:36.993789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 232/467] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:36 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:36.994942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a639f-4dc0-4d96-8c6b-9492fdcc1769 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:37.054695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:37.068246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74cf8b-8f0a-4b9e-86c1-dc1d87a1ea53 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:37.116862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff73077-5378-484b-afee-d0cebd1724ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:37.117285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a639f-4dc0-4d96-8c6b-9492fdcc1769 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:37.117480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff73077-5378-484b-afee-d0cebd1724ed None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:37.117759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff73077-5378-484b-afee-d0cebd1724ed None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:37.118031 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff73077-5378-484b-afee-d0cebd1724ed None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:37.118677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff73077-5378-484b-afee-d0cebd1724ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:37.118978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff73077-5378-484b-afee-d0cebd1724ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:37.121052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff73077-5378-484b-afee-d0cebd1724ed None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:37.134002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 236/468] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:37.135259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a27d1-b126-4890-8103-a955e61e333a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:37.135663 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a27d1-b126-4890-8103-a955e61e333a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:37.136375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a27d1-b126-4890-8103-a955e61e333a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:37.158197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-481a27d1-b126-4890-8103-a955e61e333a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:37.160825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-481a27d1-b126-4890-8103-a955e61e333a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:37.168508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3e5d3-bb3b-4bbe-a729-402baf96abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:37.169127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3e5d3-bb3b-4bbe-a729-402baf96abed None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:37.169405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3e5d3-bb3b-4bbe-a729-402baf96abed None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:37.169673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3e5d3-bb3b-4bbe-a729-402baf96abed None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:37.170323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3e5d3-bb3b-4bbe-a729-402baf96abed None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:37.170622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3e5d3-bb3b-4bbe-a729-402baf96abed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:37.172575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3e5d3-bb3b-4bbe-a729-402baf96abed None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:37.185112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 233/469] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:37.185928 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3e5d3-bb3b-4bbe-a729-402baf96abed None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:37.244371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 237/470] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:37.246207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a27d1-b126-4890-8103-a955e61e333a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:37.268299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:37.328695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3e5d3-bb3b-4bbe-a729-402baf96abed None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:37.340433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:37.363045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a27d1-b126-4890-8103-a955e61e333a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:37.367283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1cec24-b1a3-4831-970e-d14160a8ea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:37.367990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cec24-b1a3-4831-970e-d14160a8ea13 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:37.368343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cec24-b1a3-4831-970e-d14160a8ea13 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:37.368688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cec24-b1a3-4831-970e-d14160a8ea13 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:37.369469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cec24-b1a3-4831-970e-d14160a8ea13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:37.372761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1cec24-b1a3-4831-970e-d14160a8ea13 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:37.402420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee2286-6b71-46cf-b973-e3ca8a9b64a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:37.403384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee2286-6b71-46cf-b973-e3ca8a9b64a6 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:37.403946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee2286-6b71-46cf-b973-e3ca8a9b64a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:37.404390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee2286-6b71-46cf-b973-e3ca8a9b64a6 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:37.405218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee2286-6b71-46cf-b973-e3ca8a9b64a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:37.406862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee2286-6b71-46cf-b973-e3ca8a9b64a6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:37.455003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 234/471] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:41:37 2022] POST /identity/v3/users => generated 249 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:41:37.455003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe435d7c-3e66-41a2-b0bf-9aab8bd39b13 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:37.455003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe435d7c-3e66-41a2-b0bf-9aab8bd39b13 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:37.455003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe435d7c-3e66-41a2-b0bf-9aab8bd39b13 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:37.455003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 235/472] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:37.460914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e154b2-1e66-4675-8553-2d726dec2839 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:37.461225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e154b2-1e66-4675-8553-2d726dec2839 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:37.461466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e154b2-1e66-4675-8553-2d726dec2839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:37.482032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c8e154b2-1e66-4675-8553-2d726dec2839 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:37.483299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c8e154b2-1e66-4675-8553-2d726dec2839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:37.511130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 238/473] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:41:37 2022] POST /identity/v3/users => generated 249 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:41:37.547417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee2286-6b71-46cf-b973-e3ca8a9b64a6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:37.598678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 236/474] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:37.658986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:37.696581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee2286-6b71-46cf-b973-e3ca8a9b64a6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:37.737515 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe5190-061e-4ce9-b21c-162713fe84c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:37.738071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe5190-061e-4ce9-b21c-162713fe84c6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:37.738347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe5190-061e-4ce9-b21c-162713fe84c6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:37.738616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe5190-061e-4ce9-b21c-162713fe84c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:37.749707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe5190-061e-4ce9-b21c-162713fe84c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:37.751179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe5190-061e-4ce9-b21c-162713fe84c6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-95fe5190-061e-4ce9-b21c-162713fe84c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:37.765379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:37.766320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:37.767287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:37.767287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:37.767287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:37.767287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 239/475] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:37.772282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e154b2-1e66-4675-8553-2d726dec2839 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:37.880361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:37.887772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e154b2-1e66-4675-8553-2d726dec2839 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:37.923934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c20d6-5926-4999-9e51-69af5a96ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:37.924681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c20d6-5926-4999-9e51-69af5a96ab0e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:37.924923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c20d6-5926-4999-9e51-69af5a96ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:37.925195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c20d6-5926-4999-9e51-69af5a96ab0e None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:37.925841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c20d6-5926-4999-9e51-69af5a96ab0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:37.926138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c20d6-5926-4999-9e51-69af5a96ab0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:37.927593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835c20d6-5926-4999-9e51-69af5a96ab0e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:37.939680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 237/476] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:37 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:37.945202 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe5190-061e-4ce9-b21c-162713fe84c6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:37.976021 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c20d6-5926-4999-9e51-69af5a96ab0e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:38.085051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835c20d6-5926-4999-9e51-69af5a96ab0e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:38.123310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6e39e-0092-45ed-a242-6956e55f2363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:38.123930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6e39e-0092-45ed-a242-6956e55f2363 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:38.124205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6e39e-0092-45ed-a242-6956e55f2363 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:38.124549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6e39e-0092-45ed-a242-6956e55f2363 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:38.125157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6e39e-0092-45ed-a242-6956e55f2363 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:38.126600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6e39e-0092-45ed-a242-6956e55f2363 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:38.161584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.sql.core [None req-eea6e39e-0092-45ed-a242-6956e55f2363 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 04:41:38.161584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 04:41:38.161584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [parameters: {'id': '6064389b60ef4524b3719a877bb80325', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 04:41:38.161584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82575) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 04:41:38.182344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe5190-061e-4ce9-b21c-162713fe84c6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-eea6e39e-0092-45ed-a242-6956e55f2363 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application conn.query(q) Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application result.read() Aug 24 04:41:38.188679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 04:41:38.189589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application next(self.gen) Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application yield resource Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application next(self.gen) Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise self.value Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application session.commit() Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 04:41:38.190746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self.session.flush() Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application compat.raise_( Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 04:41:38.191664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 04:41:38.192572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application conn.query(q) Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application result.read() Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 04:41:38.193407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [parameters: {'id': '6064389b60ef4524b3719a877bb80325', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:38.194189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 04:41:38.195010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:38.200067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 238/477] 10.222.0.105 () {60 vars in 1136 bytes} [Wed Aug 24 04:41:37 2022] POST /identity/v3/roles => generated 144 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:38.200067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6e39e-0092-45ed-a242-6956e55f2363 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:38.236858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:38.248912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:38.272072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f119eea-aef1-4f04-8a5e-12b83c82c601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:38.272686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f119eea-aef1-4f04-8a5e-12b83c82c601 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:38.272967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f119eea-aef1-4f04-8a5e-12b83c82c601 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:38.273239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f119eea-aef1-4f04-8a5e-12b83c82c601 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:38.278122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f119eea-aef1-4f04-8a5e-12b83c82c601 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:38.279623 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f119eea-aef1-4f04-8a5e-12b83c82c601 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:38.346721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6e39e-0092-45ed-a242-6956e55f2363 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:38.384482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d574155-133a-4975-8d52-09c833415c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:38.385147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d574155-133a-4975-8d52-09c833415c37 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:38.385425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d574155-133a-4975-8d52-09c833415c37 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:38.385693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d574155-133a-4975-8d52-09c833415c37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:38.391400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 240/478] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:41:37 2022] POST /identity/v3/users => generated 250 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:41:38.408879 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d574155-133a-4975-8d52-09c833415c37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-8d574155-133a-4975-8d52-09c833415c37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:38.413969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:38.414933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:38.415807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 239/479] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:41:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:38.423672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f119eea-aef1-4f04-8a5e-12b83c82c601 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:38.567675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f119eea-aef1-4f04-8a5e-12b83c82c601 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:38.613986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff452b-7ad2-4fe0-8415-379da9c8033d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:38.614515 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff452b-7ad2-4fe0-8415-379da9c8033d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:38.614792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff452b-7ad2-4fe0-8415-379da9c8033d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:38.615123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff452b-7ad2-4fe0-8415-379da9c8033d None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:38.615908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff452b-7ad2-4fe0-8415-379da9c8033d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:38.616215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff452b-7ad2-4fe0-8415-379da9c8033d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:38.617574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff452b-7ad2-4fe0-8415-379da9c8033d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:38.627770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 241/480] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:38 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:39.092630 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347ec7c-8da9-47e9-9e15-f6faa7dba50c None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:39.093145 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347ec7c-8da9-47e9-9e15-f6faa7dba50c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:39.093145 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347ec7c-8da9-47e9-9e15-f6faa7dba50c None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:39.094890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 240/481] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:39.104524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e844c-d451-4f68-9ab0-c6758a175338 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:39.105557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e844c-d451-4f68-9ab0-c6758a175338 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:39.105702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e844c-d451-4f68-9ab0-c6758a175338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:39.131993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-424e844c-d451-4f68-9ab0-c6758a175338 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:39.133186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-424e844c-d451-4f68-9ab0-c6758a175338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:39.176386 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:39.217394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 242/482] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:39.623611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347ec7c-8da9-47e9-9e15-f6faa7dba50c None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:39.711744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:39.735437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347ec7c-8da9-47e9-9e15-f6faa7dba50c None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:39.771671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0d4dd-b93a-4005-9726-aabfdfa6de7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:39.772302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0d4dd-b93a-4005-9726-aabfdfa6de7a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:39.772635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0d4dd-b93a-4005-9726-aabfdfa6de7a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:39.772913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0d4dd-b93a-4005-9726-aabfdfa6de7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:39.784303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0d4dd-b93a-4005-9726-aabfdfa6de7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-c3a0d4dd-b93a-4005-9726-aabfdfa6de7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:39.789072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:39.790034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:39.791268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 241/483] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:39.796409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e844c-d451-4f68-9ab0-c6758a175338 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:39.906760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e4be2-f05a-41ac-a800-526bd9ad43dc None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:39.907087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e4be2-f05a-41ac-a800-526bd9ad43dc None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:39.907256 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e4be2-f05a-41ac-a800-526bd9ad43dc None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:39.908646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 242/484] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:41:39.924683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dd465-f79e-406c-b025-d398b9fdb8d9 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:39.924948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dd465-f79e-406c-b025-d398b9fdb8d9 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:39.925227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dd465-f79e-406c-b025-d398b9fdb8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:39.937918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e844c-d451-4f68-9ab0-c6758a175338 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:39.951950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-765dd465-f79e-406c-b025-d398b9fdb8d9 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:39.953347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-765dd465-f79e-406c-b025-d398b9fdb8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:39.988310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334332a-92fa-4466-bd9e-62308e158c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:39.988939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334332a-92fa-4466-bd9e-62308e158c74 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:39.989222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334332a-92fa-4466-bd9e-62308e158c74 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:39.989491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334332a-92fa-4466-bd9e-62308e158c74 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:39.990165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334332a-92fa-4466-bd9e-62308e158c74 None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:39.990454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334332a-92fa-4466-bd9e-62308e158c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:39.991847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334332a-92fa-4466-bd9e-62308e158c74 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:40.010361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 243/485] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:39 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:40.017947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334332a-92fa-4466-bd9e-62308e158c74 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:40.044913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 243/486] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:39 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:40.066538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e754a-e6b0-4df0-b02c-a32405102654 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:40.067159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e754a-e6b0-4df0-b02c-a32405102654 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:40.067683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e754a-e6b0-4df0-b02c-a32405102654 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:40.069670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 244/487] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:40 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:40.080326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430f22b-f48f-453d-9eb9-aeae5617a351 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:40.080830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430f22b-f48f-453d-9eb9-aeae5617a351 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:40.084130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430f22b-f48f-453d-9eb9-aeae5617a351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:40.094798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:40.118949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-7430f22b-f48f-453d-9eb9-aeae5617a351 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:40.122673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-7430f22b-f48f-453d-9eb9-aeae5617a351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:40.231588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334332a-92fa-4466-bd9e-62308e158c74 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:40.253863 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:40.264989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 245/488] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:40.283635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c158a-5e4d-49d0-884c-2ad7bb4d4ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:40.284570 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c158a-5e4d-49d0-884c-2ad7bb4d4ef8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:40.285077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c158a-5e4d-49d0-884c-2ad7bb4d4ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:40.285576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c158a-5e4d-49d0-884c-2ad7bb4d4ef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:40.297765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c158a-5e4d-49d0-884c-2ad7bb4d4ef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:40.299587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c158a-5e4d-49d0-884c-2ad7bb4d4ef8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-042c158a-5e4d-49d0-884c-2ad7bb4d4ef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:40.315215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:40.316374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:40.317218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:40.317218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:40.317218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:40.319309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 244/489] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:40.328102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430f22b-f48f-453d-9eb9-aeae5617a351 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:40.360436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c158a-5e4d-49d0-884c-2ad7bb4d4ef8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:40.505790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c158a-5e4d-49d0-884c-2ad7bb4d4ef8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:40.507437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:40.539090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430f22b-f48f-453d-9eb9-aeae5617a351 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:40.554057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72057dca-ab6d-445d-b48b-5578c265c0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:40.554691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72057dca-ab6d-445d-b48b-5578c265c0b9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:40.554966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72057dca-ab6d-445d-b48b-5578c265c0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:40.555254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72057dca-ab6d-445d-b48b-5578c265c0b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:40.580103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72057dca-ab6d-445d-b48b-5578c265c0b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-72057dca-ab6d-445d-b48b-5578c265c0b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:40.586174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:40.587102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:40.587912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:40.591287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 245/490] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:40.598179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72057dca-ab6d-445d-b48b-5578c265c0b9 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:40.601418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e577dd2-6d68-48ac-8944-52b35f22d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:40.602303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e577dd2-6d68-48ac-8944-52b35f22d4be None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:40.602752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e577dd2-6d68-48ac-8944-52b35f22d4be None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:40.603203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e577dd2-6d68-48ac-8944-52b35f22d4be None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:40.603995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e577dd2-6d68-48ac-8944-52b35f22d4be None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:40.604441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e577dd2-6d68-48ac-8944-52b35f22d4be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:40.606472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e577dd2-6d68-48ac-8944-52b35f22d4be None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:40.619541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 246/491] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:40.620576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e577dd2-6d68-48ac-8944-52b35f22d4be None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:40.685915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:40.710977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72057dca-ab6d-445d-b48b-5578c265c0b9 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:40.740733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e577dd2-6d68-48ac-8944-52b35f22d4be None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:40.749573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d08fcf-dd5e-4126-bde2-92a0337f587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:40.750417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d08fcf-dd5e-4126-bde2-92a0337f587f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:40.750842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d08fcf-dd5e-4126-bde2-92a0337f587f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:40.751282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d08fcf-dd5e-4126-bde2-92a0337f587f None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:40.752083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d08fcf-dd5e-4126-bde2-92a0337f587f None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:40.752553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d08fcf-dd5e-4126-bde2-92a0337f587f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:40.754176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d08fcf-dd5e-4126-bde2-92a0337f587f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:40.763837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 246/492] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:40 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:40.764899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d08fcf-dd5e-4126-bde2-92a0337f587f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:40.776012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:40.788393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9c42-04b8-4893-a769-8e6f39e6526e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:40.790323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9c42-04b8-4893-a769-8e6f39e6526e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:40.790323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9c42-04b8-4893-a769-8e6f39e6526e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:40.790323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c9c42-04b8-4893-a769-8e6f39e6526e None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:40.803751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c9c42-04b8-4893-a769-8e6f39e6526e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-400c9c42-04b8-4893-a769-8e6f39e6526e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:40.810649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:40.811816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 247/493] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:40.812686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9c42-04b8-4893-a769-8e6f39e6526e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:40.904192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d08fcf-dd5e-4126-bde2-92a0337f587f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:40.924403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c9c42-04b8-4893-a769-8e6f39e6526e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:40.943550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:40.954442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622038e-c5a5-4132-93bb-b4ef1a47c30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:40.956442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622038e-c5a5-4132-93bb-b4ef1a47c30b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:40.956442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622038e-c5a5-4132-93bb-b4ef1a47c30b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:40.956442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622038e-c5a5-4132-93bb-b4ef1a47c30b None admin] PATH_INFO: `/v3/users/nova` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:40.961746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe3d1c-b2fa-48e4-ad39-4c14c8393f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:40.962495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe3d1c-b2fa-48e4-ad39-4c14c8393f8e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:40.962895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe3d1c-b2fa-48e4-ad39-4c14c8393f8e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:40.963294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe3d1c-b2fa-48e4-ad39-4c14c8393f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:40.966306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622038e-c5a5-4132-93bb-b4ef1a47c30b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:40.968349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622038e-c5a5-4132-93bb-b4ef1a47c30b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:40.974840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe3d1c-b2fa-48e4-ad39-4c14c8393f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:40.976677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe3d1c-b2fa-48e4-ad39-4c14c8393f8e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-9622038e-c5a5-4132-93bb-b4ef1a47c30b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:40.985985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:40.987122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:40.988216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:40.988216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:40.988216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:40.988216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:40.988216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:40.988216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:40.988216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:40.988216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:40.988216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:41:40.988216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:40.991166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 247/494] 10.222.0.105 () {56 vars in 1098 bytes} [Wed Aug 24 04:41:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:40.992230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622038e-c5a5-4132-93bb-b4ef1a47c30b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-49fe3d1c-b2fa-48e4-ad39-4c14c8393f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:41.002424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:41.003494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:41.004511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:41.004511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:41.004511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:41.011055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 248/495] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:41.012180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe3d1c-b2fa-48e4-ad39-4c14c8393f8e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.074103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:41.112461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622038e-c5a5-4132-93bb-b4ef1a47c30b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:41.135595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe3d1c-b2fa-48e4-ad39-4c14c8393f8e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:41.148101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3651f5-0cd3-4075-b493-ef510ddee7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:41.148703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3651f5-0cd3-4075-b493-ef510ddee7e0 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.148980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3651f5-0cd3-4075-b493-ef510ddee7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.149251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3651f5-0cd3-4075-b493-ef510ddee7e0 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.149919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3651f5-0cd3-4075-b493-ef510ddee7e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:41.150223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3651f5-0cd3-4075-b493-ef510ddee7e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:41.151626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3651f5-0cd3-4075-b493-ef510ddee7e0 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:41.160405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 248/496] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:40 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:41.161348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3651f5-0cd3-4075-b493-ef510ddee7e0 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.171589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdad739-28ec-4ffe-a7ba-2dfee89b94ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:41.172196 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdad739-28ec-4ffe-a7ba-2dfee89b94ec None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.172475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdad739-28ec-4ffe-a7ba-2dfee89b94ec None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.172745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdad739-28ec-4ffe-a7ba-2dfee89b94ec None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.173476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdad739-28ec-4ffe-a7ba-2dfee89b94ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:41.173821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdad739-28ec-4ffe-a7ba-2dfee89b94ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:41.175277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdad739-28ec-4ffe-a7ba-2dfee89b94ec None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:41.213070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 249/497] 10.222.0.105 () {56 vars in 1138 bytes} [Wed Aug 24 04:41:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:41.214316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e47cd-3938-4b0a-b31e-1a5de16ffab7 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.214602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e47cd-3938-4b0a-b31e-1a5de16ffab7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.214875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e47cd-3938-4b0a-b31e-1a5de16ffab7 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.216943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 250/498] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:41.217956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf526-8202-471d-97b6-cb10182b6f75 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.218258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf526-8202-471d-97b6-cb10182b6f75 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.218505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecf526-8202-471d-97b6-cb10182b6f75 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.219842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 251/499] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:41 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:41.220410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf526-8202-471d-97b6-cb10182b6f75 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.248797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:41.269835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3651f5-0cd3-4075-b493-ef510ddee7e0 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:41.295782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:41.303593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb9e45-1413-4950-a273-f5eb47678900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:41.304192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb9e45-1413-4950-a273-f5eb47678900 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.304471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb9e45-1413-4950-a273-f5eb47678900 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.304741 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb9e45-1413-4950-a273-f5eb47678900 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.305376 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb9e45-1413-4950-a273-f5eb47678900 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:41.305673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb9e45-1413-4950-a273-f5eb47678900 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:41.307061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb9e45-1413-4950-a273-f5eb47678900 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:41.319206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 249/500] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:41.320166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb9e45-1413-4950-a273-f5eb47678900 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.321451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecf526-8202-471d-97b6-cb10182b6f75 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:41.356509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea54fe-a76c-4f32-af0c-987b711f8a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:41.357138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea54fe-a76c-4f32-af0c-987b711f8a2e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.357417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea54fe-a76c-4f32-af0c-987b711f8a2e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.357688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea54fe-a76c-4f32-af0c-987b711f8a2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.367499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea54fe-a76c-4f32-af0c-987b711f8a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:41.368946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea54fe-a76c-4f32-af0c-987b711f8a2e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-31ea54fe-a76c-4f32-af0c-987b711f8a2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:41.383791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:41.384895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:41.385725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:41.385725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:41.385725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:41.385725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 252/501] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:41.386080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99652add-6943-4fd3-aead-9c6795ea8092 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.386295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99652add-6943-4fd3-aead-9c6795ea8092 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.386566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99652add-6943-4fd3-aead-9c6795ea8092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.405370 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-99652add-6943-4fd3-aead-9c6795ea8092 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:41.406564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-99652add-6943-4fd3-aead-9c6795ea8092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:41.421017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb9e45-1413-4950-a273-f5eb47678900 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:41.456113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe7b1a-9615-45f7-b4f6-e7391c8947c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:41.456714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe7b1a-9615-45f7-b4f6-e7391c8947c1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.456988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe7b1a-9615-45f7-b4f6-e7391c8947c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.457258 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe7b1a-9615-45f7-b4f6-e7391c8947c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.473290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 253/502] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:41.474612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a731d-e48e-45ba-9689-6a9aff074504 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.474993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a731d-e48e-45ba-9689-6a9aff074504 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.475199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a731d-e48e-45ba-9689-6a9aff074504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.505778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c68a731d-e48e-45ba-9689-6a9aff074504 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:41.507093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c68a731d-e48e-45ba-9689-6a9aff074504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:41.510036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:41.515493 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:41.527272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe7b1a-9615-45f7-b4f6-e7391c8947c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:41.530308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe7b1a-9615-45f7-b4f6-e7391c8947c1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:41.576080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 254/503] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:41.577155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a731d-e48e-45ba-9689-6a9aff074504 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.649345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 250/504] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:41 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:41.650368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe7b1a-9615-45f7-b4f6-e7391c8947c1 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.713175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a731d-e48e-45ba-9689-6a9aff074504 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:41.749625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:41.761403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe7b1a-9615-45f7-b4f6-e7391c8947c1 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:41.761874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290e027-7d49-4ffb-9615-b439a8876670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:41.762686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290e027-7d49-4ffb-9615-b439a8876670 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.763210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290e027-7d49-4ffb-9615-b439a8876670 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.763490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290e027-7d49-4ffb-9615-b439a8876670 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.771289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290e027-7d49-4ffb-9615-b439a8876670 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:41.772719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290e027-7d49-4ffb-9615-b439a8876670 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:41.775900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-0290e027-7d49-4ffb-9615-b439a8876670 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:41.787261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:41.794583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 255/505] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:41.795611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290e027-7d49-4ffb-9615-b439a8876670 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.805453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d57b9-b78d-4cbc-ab6c-a358d1acecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:41.806066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d57b9-b78d-4cbc-ab6c-a358d1acecaf None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.806343 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d57b9-b78d-4cbc-ab6c-a358d1acecaf None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.806611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d57b9-b78d-4cbc-ab6c-a358d1acecaf None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.807284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d57b9-b78d-4cbc-ab6c-a358d1acecaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:41.807588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d57b9-b78d-4cbc-ab6c-a358d1acecaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:41.808941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d57b9-b78d-4cbc-ab6c-a358d1acecaf None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:41.821064 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 251/506] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:41.821064 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d57b9-b78d-4cbc-ab6c-a358d1acecaf None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.906239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290e027-7d49-4ffb-9615-b439a8876670 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:41.922348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d57b9-b78d-4cbc-ab6c-a358d1acecaf None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:41.940553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45dd74-6d27-4151-8dca-32df821606cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:41.941183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45dd74-6d27-4151-8dca-32df821606cc None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.941464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45dd74-6d27-4151-8dca-32df821606cc None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.942005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45dd74-6d27-4151-8dca-32df821606cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.952505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45dd74-6d27-4151-8dca-32df821606cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:41.953957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45dd74-6d27-4151-8dca-32df821606cc None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:41.956284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583e256-128b-439b-b0fe-02328517b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:41.956901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583e256-128b-439b-b0fe-02328517b86e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:41.957180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583e256-128b-439b-b0fe-02328517b86e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:41.957448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583e256-128b-439b-b0fe-02328517b86e None admin] PATH_INFO: `/v3/roles/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:41.967777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583e256-128b-439b-b0fe-02328517b86e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-1a45dd74-6d27-4151-8dca-32df821606cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:41.967892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:41.968896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:41.969901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:41.969901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:41.969901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:41.969901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 256/507] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:41.969901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45dd74-6d27-4151-8dca-32df821606cc None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-3583e256-128b-439b-b0fe-02328517b86e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:41.975296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:41.976403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 252/508] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:41.977204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583e256-128b-439b-b0fe-02328517b86e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:41.988652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:42.027498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:42.074725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45dd74-6d27-4151-8dca-32df821606cc None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.077073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583e256-128b-439b-b0fe-02328517b86e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.109294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf4ccf-9295-46ae-a4fd-9b3f0120301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.109978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf4ccf-9295-46ae-a4fd-9b3f0120301c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.110259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf4ccf-9295-46ae-a4fd-9b3f0120301c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.110535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf4ccf-9295-46ae-a4fd-9b3f0120301c None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.110775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d6637-1a06-4c62-a212-97404e398378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.111291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf4ccf-9295-46ae-a4fd-9b3f0120301c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:42.111435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d6637-1a06-4c62-a212-97404e398378 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.111583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf4ccf-9295-46ae-a4fd-9b3f0120301c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.111712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d6637-1a06-4c62-a212-97404e398378 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.111981 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d6637-1a06-4c62-a212-97404e398378 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.113013 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf4ccf-9295-46ae-a4fd-9b3f0120301c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.119332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d6637-1a06-4c62-a212-97404e398378 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.120772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d6637-1a06-4c62-a212-97404e398378 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.130313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 257/509] 10.222.0.105 () {56 vars in 1142 bytes} [Wed Aug 24 04:41:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:42.131081 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf4ccf-9295-46ae-a4fd-9b3f0120301c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-1a9d6637-1a06-4c62-a212-97404e398378 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:42.132508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:42.133370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:42.134318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 253/510] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:42.135067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d6637-1a06-4c62-a212-97404e398378 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:42.205462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:42.236452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf4ccf-9295-46ae-a4fd-9b3f0120301c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.239442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d6637-1a06-4c62-a212-97404e398378 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.247339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:42.270867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3b086-14b1-4f36-8736-5f805901faca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.271520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3b086-14b1-4f36-8736-5f805901faca None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.271800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3b086-14b1-4f36-8736-5f805901faca None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.272074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3b086-14b1-4f36-8736-5f805901faca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.273257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fedf9-faed-493e-9e78-9367c195a08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.273858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fedf9-faed-493e-9e78-9367c195a08d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.274138 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fedf9-faed-493e-9e78-9367c195a08d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.274405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fedf9-faed-493e-9e78-9367c195a08d None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.275056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fedf9-faed-493e-9e78-9367c195a08d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:42.275360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fedf9-faed-493e-9e78-9367c195a08d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.276745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fedf9-faed-493e-9e78-9367c195a08d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.282954 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3b086-14b1-4f36-8736-5f805901faca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.284417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3b086-14b1-4f36-8736-5f805901faca None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.288224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 254/511] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:42.292719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fedf9-faed-493e-9e78-9367c195a08d None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-6bc3b086-14b1-4f36-8736-5f805901faca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:42.301301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:42.302299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:42.303224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:42.303224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:42.303224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:42.303224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 258/512] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:42.303224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3b086-14b1-4f36-8736-5f805901faca None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:42.388618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fedf9-faed-493e-9e78-9367c195a08d None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.402850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3b086-14b1-4f36-8736-5f805901faca None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.414667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:42.422007 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8a9fd-4222-46f1-b7a5-124e88c3409c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.424457 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8a9fd-4222-46f1-b7a5-124e88c3409c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.424457 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8a9fd-4222-46f1-b7a5-124e88c3409c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.424457 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8a9fd-4222-46f1-b7a5-124e88c3409c None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.424457 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8a9fd-4222-46f1-b7a5-124e88c3409c None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:42.424457 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8a9fd-4222-46f1-b7a5-124e88c3409c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.425464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8a9fd-4222-46f1-b7a5-124e88c3409c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.433338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 255/513] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:42 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:42.434133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8a9fd-4222-46f1-b7a5-124e88c3409c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:42.436283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4673804-b256-49ec-bd0b-d7cc340e75e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.436916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4673804-b256-49ec-bd0b-d7cc340e75e1 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.437198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4673804-b256-49ec-bd0b-d7cc340e75e1 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.437473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4673804-b256-49ec-bd0b-d7cc340e75e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.468032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:42.506107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4673804-b256-49ec-bd0b-d7cc340e75e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.507588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4673804-b256-49ec-bd0b-d7cc340e75e1 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.537250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8a9fd-4222-46f1-b7a5-124e88c3409c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.570279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38c1a4-df4d-4052-a755-61c0705a5a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.570875 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38c1a4-df4d-4052-a755-61c0705a5a34 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.571182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38c1a4-df4d-4052-a755-61c0705a5a34 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.571451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38c1a4-df4d-4052-a755-61c0705a5a34 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.572163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38c1a4-df4d-4052-a755-61c0705a5a34 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:42.572466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38c1a4-df4d-4052-a755-61c0705a5a34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.573899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38c1a4-df4d-4052-a755-61c0705a5a34 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.605477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 259/514] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:42 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:42.606317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4673804-b256-49ec-bd0b-d7cc340e75e1 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:42.618348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 256/515] 10.222.0.105 () {56 vars in 1142 bytes} [Wed Aug 24 04:41:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:42.618348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38c1a4-df4d-4052-a755-61c0705a5a34 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:42.662766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:42.670466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:42.707637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4673804-b256-49ec-bd0b-d7cc340e75e1 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.712180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38c1a4-df4d-4052-a755-61c0705a5a34 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.740890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc01bb0-91d3-4fc2-992a-f536d68fddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.741449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc01bb0-91d3-4fc2-992a-f536d68fddb4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.741726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc01bb0-91d3-4fc2-992a-f536d68fddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.741999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc01bb0-91d3-4fc2-992a-f536d68fddb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.745419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234fc92-4b4c-4c7b-ab46-e90cdd24eb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.746019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234fc92-4b4c-4c7b-ab46-e90cdd24eb60 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.746296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234fc92-4b4c-4c7b-ab46-e90cdd24eb60 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.746563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234fc92-4b4c-4c7b-ab46-e90cdd24eb60 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.747450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234fc92-4b4c-4c7b-ab46-e90cdd24eb60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:42.747915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234fc92-4b4c-4c7b-ab46-e90cdd24eb60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.748756 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc01bb0-91d3-4fc2-992a-f536d68fddb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.749289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234fc92-4b4c-4c7b-ab46-e90cdd24eb60 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.750169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc01bb0-91d3-4fc2-992a-f536d68fddb4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.760732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 257/516] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:42.761593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234fc92-4b4c-4c7b-ab46-e90cdd24eb60 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-cbc01bb0-91d3-4fc2-992a-f536d68fddb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:42.761784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:42.762591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:42.764296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:42.764296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:42.764296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:42.764296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:42.764296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:42.764296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:42.764296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:42.764296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:42.764296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 260/517] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:42.765291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc01bb0-91d3-4fc2-992a-f536d68fddb4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:42.859038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234fc92-4b4c-4c7b-ab46-e90cdd24eb60 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.863222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc01bb0-91d3-4fc2-992a-f536d68fddb4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:42.868169 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:42.885637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:42.892648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f84eb7-de14-46b3-95bb-1e3b94aed6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.893262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f84eb7-de14-46b3-95bb-1e3b94aed6b4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.893537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f84eb7-de14-46b3-95bb-1e3b94aed6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.893804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f84eb7-de14-46b3-95bb-1e3b94aed6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.897529 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddf32d-905a-4c88-bbb1-4aa73418acda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:42.898138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddf32d-905a-4c88-bbb1-4aa73418acda None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:42.898412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddf32d-905a-4c88-bbb1-4aa73418acda None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:42.898676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ddf32d-905a-4c88-bbb1-4aa73418acda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:42.903349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f84eb7-de14-46b3-95bb-1e3b94aed6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.905712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f84eb7-de14-46b3-95bb-1e3b94aed6b4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.909625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddf32d-905a-4c88-bbb1-4aa73418acda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:42.911102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ddf32d-905a-4c88-bbb1-4aa73418acda None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-a6f84eb7-de14-46b3-95bb-1e3b94aed6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:42.920586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:42.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:42.928481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:42.928481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:42.928481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:42.928481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 258/518] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:42.928481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f84eb7-de14-46b3-95bb-1e3b94aed6b4 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-69ddf32d-905a-4c88-bbb1-4aa73418acda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:42.928747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:42.929561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:42.930439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:42.930439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:42.930439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:42.930439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 261/519] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:42.930439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddf32d-905a-4c88-bbb1-4aa73418acda None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:43.031165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f84eb7-de14-46b3-95bb-1e3b94aed6b4 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:43.031371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ddf32d-905a-4c88-bbb1-4aa73418acda None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:43.059417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08b0d7-41cb-4f2a-9d79-4eeeb1656141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:43.065466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08b0d7-41cb-4f2a-9d79-4eeeb1656141 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:43.065466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08b0d7-41cb-4f2a-9d79-4eeeb1656141 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:43.065466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08b0d7-41cb-4f2a-9d79-4eeeb1656141 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:43.065861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c092ca3-9856-4003-9a95-b5fd5ff0bf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:43.065861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c092ca3-9856-4003-9a95-b5fd5ff0bf81 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:43.065861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c092ca3-9856-4003-9a95-b5fd5ff0bf81 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:43.065861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c092ca3-9856-4003-9a95-b5fd5ff0bf81 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:43.066162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c092ca3-9856-4003-9a95-b5fd5ff0bf81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:43.066507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c092ca3-9856-4003-9a95-b5fd5ff0bf81 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:43.089368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:43.127651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08b0d7-41cb-4f2a-9d79-4eeeb1656141 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:43.129100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08b0d7-41cb-4f2a-9d79-4eeeb1656141 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:43.178967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 262/520] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:41:42 2022] POST /identity/v3/users => generated 248 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:41:43.179913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c092ca3-9856-4003-9a95-b5fd5ff0bf81 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:43.232764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 259/521] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:42 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:43.234238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08b0d7-41cb-4f2a-9d79-4eeeb1656141 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:43.235223 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:43.315054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:43.316685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c092ca3-9856-4003-9a95-b5fd5ff0bf81 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:43.362991 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76634a6b-fa47-4dcd-90d5-d7eaaf258ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:43.363722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76634a6b-fa47-4dcd-90d5-d7eaaf258ff4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:43.364003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76634a6b-fa47-4dcd-90d5-d7eaaf258ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:43.364273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76634a6b-fa47-4dcd-90d5-d7eaaf258ff4 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:43.364624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08b0d7-41cb-4f2a-9d79-4eeeb1656141 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:43.364974 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76634a6b-fa47-4dcd-90d5-d7eaaf258ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:43.365276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76634a6b-fa47-4dcd-90d5-d7eaaf258ff4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:43.366707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76634a6b-fa47-4dcd-90d5-d7eaaf258ff4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:43.386677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 263/522] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:43.387651 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76634a6b-fa47-4dcd-90d5-d7eaaf258ff4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:43.401578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bf1af-6357-45fa-9d90-6bf69063b758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:43.402182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bf1af-6357-45fa-9d90-6bf69063b758 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:43.402461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bf1af-6357-45fa-9d90-6bf69063b758 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:43.402730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694bf1af-6357-45fa-9d90-6bf69063b758 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:43.403403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bf1af-6357-45fa-9d90-6bf69063b758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:43.403897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bf1af-6357-45fa-9d90-6bf69063b758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:43.405334 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694bf1af-6357-45fa-9d90-6bf69063b758 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:43.417098 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 260/523] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:43.422639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bf1af-6357-45fa-9d90-6bf69063b758 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:43.494901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:43.505801 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76634a6b-fa47-4dcd-90d5-d7eaaf258ff4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:43.524662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694bf1af-6357-45fa-9d90-6bf69063b758 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:43.542629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35972d41-eaf5-4a7e-8232-d36c52e08d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:43.543259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35972d41-eaf5-4a7e-8232-d36c52e08d7e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:43.543540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35972d41-eaf5-4a7e-8232-d36c52e08d7e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:43.543811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35972d41-eaf5-4a7e-8232-d36c52e08d7e None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:43.544532 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35972d41-eaf5-4a7e-8232-d36c52e08d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:43.544870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35972d41-eaf5-4a7e-8232-d36c52e08d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:43.546236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35972d41-eaf5-4a7e-8232-d36c52e08d7e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:43.546575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:43.558236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 264/524] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:43.560101 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35972d41-eaf5-4a7e-8232-d36c52e08d7e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:43.560410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c6dae-6b10-4e64-a637-314600481569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:43.561119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c6dae-6b10-4e64-a637-314600481569 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:43.561503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c6dae-6b10-4e64-a637-314600481569 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:43.561823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c6dae-6b10-4e64-a637-314600481569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:43.572654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c6dae-6b10-4e64-a637-314600481569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:43.574811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c6dae-6b10-4e64-a637-314600481569 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-9c7c6dae-6b10-4e64-a637-314600481569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:43.591109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:43.593126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:43.594009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:43.594009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:43.594009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:43.594009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 261/525] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:43.594009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c6dae-6b10-4e64-a637-314600481569 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:43.677624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35972d41-eaf5-4a7e-8232-d36c52e08d7e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:43.701386 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c6dae-6b10-4e64-a637-314600481569 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:43.714429 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e5285-7d50-4384-b1ad-6535d6472f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:43.715042 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e5285-7d50-4384-b1ad-6535d6472f05 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:43.715334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e5285-7d50-4384-b1ad-6535d6472f05 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:43.715606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e5285-7d50-4384-b1ad-6535d6472f05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:43.716363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e5285-7d50-4384-b1ad-6535d6472f05 None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=3c822c8f23b74bae84769046fdb85db4) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:43.716680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e5285-7d50-4384-b1ad-6535d6472f05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:43.718193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e5285-7d50-4384-b1ad-6535d6472f05 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:43.735391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 265/526] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:43 2022] GET /identity/v3/role_assignments?user.id=3c822c8f23b74bae84769046fdb85db4&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:43.736318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e5285-7d50-4384-b1ad-6535d6472f05 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:43.737677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60cdcc-c4c5-4f13-a60b-98b9a37ee450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:43.738329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60cdcc-c4c5-4f13-a60b-98b9a37ee450 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:43.738611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60cdcc-c4c5-4f13-a60b-98b9a37ee450 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:43.738878 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60cdcc-c4c5-4f13-a60b-98b9a37ee450 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:43.748328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60cdcc-c4c5-4f13-a60b-98b9a37ee450 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:43.749822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60cdcc-c4c5-4f13-a60b-98b9a37ee450 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:43.761117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-fe60cdcc-c4c5-4f13-a60b-98b9a37ee450 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:43.763011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:43.764552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:43.765469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:43.765469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:43.765469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:43.765469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:43.765469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:43.765469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:43.765469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:43.765469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:43.765469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 04:41:43.765469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:43.769662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 262/527] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:43.770701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60cdcc-c4c5-4f13-a60b-98b9a37ee450 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:43.808541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:43.873671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e5285-7d50-4384-b1ad-6535d6472f05 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:43.889885 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60cdcc-c4c5-4f13-a60b-98b9a37ee450 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:43.912555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb91ea-59b6-45e4-953b-b4894a3bc706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:43.913792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb91ea-59b6-45e4-953b-b4894a3bc706 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:43.914542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb91ea-59b6-45e4-953b-b4894a3bc706 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:43.915457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb91ea-59b6-45e4-953b-b4894a3bc706 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:43.925183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb91ea-59b6-45e4-953b-b4894a3bc706 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:43.927097 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec68e5c-4d54-4f70-b11b-c678a688d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:43.927471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eb91ea-59b6-45e4-953b-b4894a3bc706 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:43.927701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec68e5c-4d54-4f70-b11b-c678a688d755 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:43.928023 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec68e5c-4d54-4f70-b11b-c678a688d755 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:43.928295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec68e5c-4d54-4f70-b11b-c678a688d755 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:43.928919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec68e5c-4d54-4f70-b11b-c678a688d755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:43.929220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec68e5c-4d54-4f70-b11b-c678a688d755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:43.930684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec68e5c-4d54-4f70-b11b-c678a688d755 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:43.943429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 263/528] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:43.944093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec68e5c-4d54-4f70-b11b-c678a688d755 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-81eb91ea-59b6-45e4-953b-b4894a3bc706 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:43.944284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:43.945111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:43.945961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:43.945961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:43.945961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:43.945961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:43.945961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:43.945961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:43.945961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:43.945961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:43.948312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 266/529] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:43.949611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb91ea-59b6-45e4-953b-b4894a3bc706 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:44.050124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:44.050682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:44.060722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec68e5c-4d54-4f70-b11b-c678a688d755 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:44.087160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb91ea-59b6-45e4-953b-b4894a3bc706 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:44.103734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7578e-ff89-4afd-8a20-c6e373a82a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:44.104510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7578e-ff89-4afd-8a20-c6e373a82a28 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:44.104860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7578e-ff89-4afd-8a20-c6e373a82a28 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:44.105205 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7578e-ff89-4afd-8a20-c6e373a82a28 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:44.106268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7578e-ff89-4afd-8a20-c6e373a82a28 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:44.107547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7578e-ff89-4afd-8a20-c6e373a82a28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:44.109050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f7578e-ff89-4afd-8a20-c6e373a82a28 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:44.131334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef8e9-ed4f-43dc-ae64-d42f715158f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:44.131970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef8e9-ed4f-43dc-ae64-d42f715158f7 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:44.132338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 264/530] 10.222.0.105 () {56 vars in 1144 bytes} [Wed Aug 24 04:41:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:44.133400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7578e-ff89-4afd-8a20-c6e373a82a28 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:44.137031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef8e9-ed4f-43dc-ae64-d42f715158f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:44.137414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef8e9-ed4f-43dc-ae64-d42f715158f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:44.152522 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ef8e9-ed4f-43dc-ae64-d42f715158f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:44.155525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ef8e9-ed4f-43dc-ae64-d42f715158f7 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-622ef8e9-ed4f-43dc-ae64-d42f715158f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:44.171152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:44.172031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:44.172911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:44.172911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:44.172911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:44.172911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:44.172911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:44.172911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:44.172911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:44.172911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:44.172911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 267/531] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:44.173376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef8e9-ed4f-43dc-ae64-d42f715158f7 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:44.251089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7578e-ff89-4afd-8a20-c6e373a82a28 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:44.295742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb599c87-8498-48fc-b1e2-2515cd59694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:44.296359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb599c87-8498-48fc-b1e2-2515cd59694f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:44.296639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb599c87-8498-48fc-b1e2-2515cd59694f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:44.297030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb599c87-8498-48fc-b1e2-2515cd59694f None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:44.297814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb599c87-8498-48fc-b1e2-2515cd59694f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:44.298123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb599c87-8498-48fc-b1e2-2515cd59694f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:44.299593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb599c87-8498-48fc-b1e2-2515cd59694f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:44.307104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef8e9-ed4f-43dc-ae64-d42f715158f7 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:44.324705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 265/532] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:44.325776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb599c87-8498-48fc-b1e2-2515cd59694f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:44.340518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:44.352998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df0df8-61c7-4da2-9155-8a295651c791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:44.354607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df0df8-61c7-4da2-9155-8a295651c791 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:44.355058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df0df8-61c7-4da2-9155-8a295651c791 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:44.355712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2df0df8-61c7-4da2-9155-8a295651c791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:44.359925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:44.433929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df0df8-61c7-4da2-9155-8a295651c791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:44.435499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2df0df8-61c7-4da2-9155-8a295651c791 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:44.452390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb599c87-8498-48fc-b1e2-2515cd59694f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:44.498221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd1ab8-8666-43a8-8a2b-dc107608c805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:44.499196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd1ab8-8666-43a8-8a2b-dc107608c805 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:44.499915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd1ab8-8666-43a8-8a2b-dc107608c805 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:44.500590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd1ab8-8666-43a8-8a2b-dc107608c805 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:44.501434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd1ab8-8666-43a8-8a2b-dc107608c805 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:44.502017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd1ab8-8666-43a8-8a2b-dc107608c805 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:44.504237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd1ab8-8666-43a8-8a2b-dc107608c805 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:44.519306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 266/533] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:44.520309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd1ab8-8666-43a8-8a2b-dc107608c805 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:44.549195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 268/534] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:44 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:44.549195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df0df8-61c7-4da2-9155-8a295651c791 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:44.584548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:44.613192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:44.648296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd1ab8-8666-43a8-8a2b-dc107608c805 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:44.678116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2df0df8-61c7-4da2-9155-8a295651c791 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:44.688348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972906c-f2c6-42bc-a002-903f9d3a0dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:44.689194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972906c-f2c6-42bc-a002-903f9d3a0dd5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:44.689678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972906c-f2c6-42bc-a002-903f9d3a0dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:44.690194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972906c-f2c6-42bc-a002-903f9d3a0dd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:44.691162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972906c-f2c6-42bc-a002-903f9d3a0dd5 None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=f1d62a4c07c4402ea5803f6e68fb1d0b) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:44.691698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972906c-f2c6-42bc-a002-903f9d3a0dd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:44.693554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972906c-f2c6-42bc-a002-903f9d3a0dd5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:44.707478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 267/535] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:44 2022] GET /identity/v3/role_assignments?user.id=f1d62a4c07c4402ea5803f6e68fb1d0b&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:44.708914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972906c-f2c6-42bc-a002-903f9d3a0dd5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:44.719340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24662f-6a69-4e60-857e-faa6949901b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:44.719955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24662f-6a69-4e60-857e-faa6949901b6 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:44.720227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24662f-6a69-4e60-857e-faa6949901b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:44.720493 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f24662f-6a69-4e60-857e-faa6949901b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:44.721229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24662f-6a69-4e60-857e-faa6949901b6 None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=29af87e1e05b4880b9ec96865994b635) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:44.721533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24662f-6a69-4e60-857e-faa6949901b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:44.722973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f24662f-6a69-4e60-857e-faa6949901b6 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:44.731101 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 269/536] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:44 2022] GET /identity/v3/role_assignments?user.id=29af87e1e05b4880b9ec96865994b635&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:44.870190 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972906c-f2c6-42bc-a002-903f9d3a0dd5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:44.923824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a018122-4135-442b-b029-bf093fae3d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:44.924838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a018122-4135-442b-b029-bf093fae3d79 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:44.925298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a018122-4135-442b-b029-bf093fae3d79 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:44.925742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a018122-4135-442b-b029-bf093fae3d79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:44.928185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:44.945265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a018122-4135-442b-b029-bf093fae3d79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:44.945265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a018122-4135-442b-b029-bf093fae3d79 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-0a018122-4135-442b-b029-bf093fae3d79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:44.962636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:44.963633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:44.964909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:44.964909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:44.964909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:44.964909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 268/537] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:44.970262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24662f-6a69-4e60-857e-faa6949901b6 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:45.101351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:45.105631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f24662f-6a69-4e60-857e-faa6949901b6 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:45.148326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5c601-b7c2-4b22-be43-63391ac3a18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:45.148937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5c601-b7c2-4b22-be43-63391ac3a18b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:45.149209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5c601-b7c2-4b22-be43-63391ac3a18b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:45.149514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5c601-b7c2-4b22-be43-63391ac3a18b None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:45.150169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5c601-b7c2-4b22-be43-63391ac3a18b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:45.150466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5c601-b7c2-4b22-be43-63391ac3a18b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:45.151961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5c601-b7c2-4b22-be43-63391ac3a18b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:45.165803 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 270/538] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:45.173752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a018122-4135-442b-b029-bf093fae3d79 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:45.215056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1efee7-4748-4aca-86fe-261fd6e412da None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:45.215350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1efee7-4748-4aca-86fe-261fd6e412da None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:45.215566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1efee7-4748-4aca-86fe-261fd6e412da None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:45.217198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 271/539] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:45.222260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac718b14-5f07-4921-b208-d8b333b3ea83 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:45.222601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac718b14-5f07-4921-b208-d8b333b3ea83 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:45.222881 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac718b14-5f07-4921-b208-d8b333b3ea83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:45.241034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ac718b14-5f07-4921-b208-d8b333b3ea83 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:45.242229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ac718b14-5f07-4921-b208-d8b333b3ea83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:45.329686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a018122-4135-442b-b029-bf093fae3d79 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:45.348369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 272/540] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:45.415711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac4a2b-a0cd-43e2-b663-5da14df01d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:45.416344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac4a2b-a0cd-43e2-b663-5da14df01d04 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:45.416620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac4a2b-a0cd-43e2-b663-5da14df01d04 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:45.416885 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac4a2b-a0cd-43e2-b663-5da14df01d04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:45.428070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac4a2b-a0cd-43e2-b663-5da14df01d04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:45.429519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac4a2b-a0cd-43e2-b663-5da14df01d04 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-faac4a2b-a0cd-43e2-b663-5da14df01d04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:45.448279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:45.449172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:45.450306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:45.450306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:45.450306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:45.450306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:45.450306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:45.450306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:45.450306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:45.450306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:45.450306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 269/541] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:45.456146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac718b14-5f07-4921-b208-d8b333b3ea83 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:45.581890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:45.598679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac718b14-5f07-4921-b208-d8b333b3ea83 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:45.641446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fe68c-5343-4340-baa4-627299dd0551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:45.642062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fe68c-5343-4340-baa4-627299dd0551 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:45.642341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fe68c-5343-4340-baa4-627299dd0551 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:45.642607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fe68c-5343-4340-baa4-627299dd0551 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:45.643341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fe68c-5343-4340-baa4-627299dd0551 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:45.643634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fe68c-5343-4340-baa4-627299dd0551 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:45.645053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3fe68c-5343-4340-baa4-627299dd0551 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:45.663681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac4a2b-a0cd-43e2-b663-5da14df01d04 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:45.665394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 273/542] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:45.673315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fe68c-5343-4340-baa4-627299dd0551 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:45.746586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:45.839715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac4a2b-a0cd-43e2-b663-5da14df01d04 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:45.852853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fe68c-5343-4340-baa4-627299dd0551 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:45.904127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d0e62-a254-4e2e-85ae-fba56900ddb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:45.905319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d0e62-a254-4e2e-85ae-fba56900ddb4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:45.905600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d0e62-a254-4e2e-85ae-fba56900ddb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:45.905869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d0e62-a254-4e2e-85ae-fba56900ddb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:45.913300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b787a-fd6e-451f-8fc9-fc954e0d1079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:45.913911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b787a-fd6e-451f-8fc9-fc954e0d1079 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:45.914246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b787a-fd6e-451f-8fc9-fc954e0d1079 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:45.914522 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b787a-fd6e-451f-8fc9-fc954e0d1079 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:45.915996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b787a-fd6e-451f-8fc9-fc954e0d1079 None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=44d2b4015ac74b8b8d0a674419f34cf7) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:45.916307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b787a-fd6e-451f-8fc9-fc954e0d1079 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:45.917737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b787a-fd6e-451f-8fc9-fc954e0d1079 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:45.926802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d0e62-a254-4e2e-85ae-fba56900ddb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:45.932076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 274/543] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:45 2022] GET /identity/v3/role_assignments?user.id=44d2b4015ac74b8b8d0a674419f34cf7&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-a74d0e62-a254-4e2e-85ae-fba56900ddb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:45.934035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:45.934898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:45.935780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:45.939526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 270/544] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:45.955160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b787a-fd6e-451f-8fc9-fc954e0d1079 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:45.974310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:46.104744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b787a-fd6e-451f-8fc9-fc954e0d1079 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:46.121413 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a705-76fe-4e2f-9549-889fd16f21b1 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:46.121732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a705-76fe-4e2f-9549-889fd16f21b1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:46.121971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b8a705-76fe-4e2f-9549-889fd16f21b1 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:46.123641 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 271/545] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:41:46.131298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88d6aa-ca01-420c-afe2-5192125cbe55 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:46.131610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88d6aa-ca01-420c-afe2-5192125cbe55 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:46.131853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88d6aa-ca01-420c-afe2-5192125cbe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:46.150024 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e4ab6-ac75-44f0-9ccf-f8d2e88cc8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:46.150703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e4ab6-ac75-44f0-9ccf-f8d2e88cc8be None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:46.150986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e4ab6-ac75-44f0-9ccf-f8d2e88cc8be None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:46.151284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e4ab6-ac75-44f0-9ccf-f8d2e88cc8be None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:46.152007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e4ab6-ac75-44f0-9ccf-f8d2e88cc8be None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:46.152307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e4ab6-ac75-44f0-9ccf-f8d2e88cc8be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:46.153679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e4ab6-ac75-44f0-9ccf-f8d2e88cc8be None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:46.160200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-fa88d6aa-ca01-420c-afe2-5192125cbe55 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:46.161414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-fa88d6aa-ca01-420c-afe2-5192125cbe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:46.167399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 275/546] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:45 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:46.168797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e4ab6-ac75-44f0-9ccf-f8d2e88cc8be None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:46.253934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 272/547] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 04:41:46.263620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:46.309537 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e4ab6-ac75-44f0-9ccf-f8d2e88cc8be None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:46.366218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f347e4-d8d7-474d-a85c-268c54fe49c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:46.366749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f347e4-d8d7-474d-a85c-268c54fe49c7 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:46.367058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f347e4-d8d7-474d-a85c-268c54fe49c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:46.367311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f347e4-d8d7-474d-a85c-268c54fe49c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:46.379432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f347e4-d8d7-474d-a85c-268c54fe49c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:46.380916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f347e4-d8d7-474d-a85c-268c54fe49c7 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-84f347e4-d8d7-474d-a85c-268c54fe49c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:46.395261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:46.397600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:46.399603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:46.399603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:46.399603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:46.399603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 276/548] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:46.405833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88d6aa-ca01-420c-afe2-5192125cbe55 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:46.456154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:46.557317 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88d6aa-ca01-420c-afe2-5192125cbe55 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:46.560516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f347e4-d8d7-474d-a85c-268c54fe49c7 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:46.626438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c7a26-4661-46b9-8313-79314a46b4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:46.627013 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7a26-4661-46b9-8313-79314a46b4c8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:46.627809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7a26-4661-46b9-8313-79314a46b4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:46.627809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c7a26-4661-46b9-8313-79314a46b4c8 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:46.628316 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c7a26-4661-46b9-8313-79314a46b4c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:46.628554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c7a26-4661-46b9-8313-79314a46b4c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:46.629880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c7a26-4661-46b9-8313-79314a46b4c8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:46.642111 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 273/549] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:46.647546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c7a26-4661-46b9-8313-79314a46b4c8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:46.766331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f347e4-d8d7-474d-a85c-268c54fe49c7 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:46.815573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:46.822262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c7a26-4661-46b9-8313-79314a46b4c8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:46.840805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:46.863425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2324918-2c28-4e94-9c3c-1a35286b5e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:46.864075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2324918-2c28-4e94-9c3c-1a35286b5e93 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:46.864386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2324918-2c28-4e94-9c3c-1a35286b5e93 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:46.864641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2324918-2c28-4e94-9c3c-1a35286b5e93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:46.865900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c020c8d-d746-4bd2-a619-42906b82738b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:46.866519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c020c8d-d746-4bd2-a619-42906b82738b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:46.867056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c020c8d-d746-4bd2-a619-42906b82738b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:46.867056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c020c8d-d746-4bd2-a619-42906b82738b None admin] PATH_INFO: `/v3/users/swift` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:46.882399 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c020c8d-d746-4bd2-a619-42906b82738b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:46.883835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c020c8d-d746-4bd2-a619-42906b82738b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:46.888598 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2324918-2c28-4e94-9c3c-1a35286b5e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-e2324918-2c28-4e94-9c3c-1a35286b5e93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:46.894093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:46.896688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:46.898263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 277/550] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:46.903291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2324918-2c28-4e94-9c3c-1a35286b5e93 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-9c020c8d-d746-4bd2-a619-42906b82738b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:46.941025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:46.945396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 274/551] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:41:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:46.951175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c020c8d-d746-4bd2-a619-42906b82738b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:47.016551 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2324918-2c28-4e94-9c3c-1a35286b5e93 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:47.055205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2dd5f-7388-464a-bc9f-cb8735ebe00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:47.055904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2dd5f-7388-464a-bc9f-cb8735ebe00e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.056159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2dd5f-7388-464a-bc9f-cb8735ebe00e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.056402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2dd5f-7388-464a-bc9f-cb8735ebe00e None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.056977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2dd5f-7388-464a-bc9f-cb8735ebe00e None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:47.057276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2dd5f-7388-464a-bc9f-cb8735ebe00e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:47.058647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2dd5f-7388-464a-bc9f-cb8735ebe00e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:47.069060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 278/552] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:46 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:47.075051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2dd5f-7388-464a-bc9f-cb8735ebe00e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:47.131780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:47.196219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2dd5f-7388-464a-bc9f-cb8735ebe00e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:47.235796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc2d4-c937-499a-820d-90c1b2979694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:47.236432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc2d4-c937-499a-820d-90c1b2979694 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.236719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc2d4-c937-499a-820d-90c1b2979694 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.236955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619cc2d4-c937-499a-820d-90c1b2979694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.249531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cc2d4-c937-499a-820d-90c1b2979694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:47.251085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619cc2d4-c937-499a-820d-90c1b2979694 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-619cc2d4-c937-499a-820d-90c1b2979694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:47.266978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:47.270734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:47.274219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:47.274219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:47.274219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:47.274219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 279/553] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:47.274219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc2d4-c937-499a-820d-90c1b2979694 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:47.303307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c020c8d-d746-4bd2-a619-42906b82738b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:47.368410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69966a6-0009-4479-bf68-e0aa4f9c2b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:47.369948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69966a6-0009-4479-bf68-e0aa4f9c2b65 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.370689 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69966a6-0009-4479-bf68-e0aa4f9c2b65 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.371490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69966a6-0009-4479-bf68-e0aa4f9c2b65 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.373111 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69966a6-0009-4479-bf68-e0aa4f9c2b65 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:47.374561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69966a6-0009-4479-bf68-e0aa4f9c2b65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:47.377564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69966a6-0009-4479-bf68-e0aa4f9c2b65 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:47.387459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:47.391456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619cc2d4-c937-499a-820d-90c1b2979694 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:47.413457 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 275/554] 10.222.0.105 () {56 vars in 1140 bytes} [Wed Aug 24 04:41:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:47.416858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14e421-9264-471e-94be-4639530c91b5 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.417690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14e421-9264-471e-94be-4639530c91b5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.418482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14e421-9264-471e-94be-4639530c91b5 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.427314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 276/555] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:47 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:47.427314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14e421-9264-471e-94be-4639530c91b5 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:47.437113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86ae99-9dfa-4262-b603-fd691242c183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:47.437732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86ae99-9dfa-4262-b603-fd691242c183 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.438025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86ae99-9dfa-4262-b603-fd691242c183 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.438268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86ae99-9dfa-4262-b603-fd691242c183 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.438915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86ae99-9dfa-4262-b603-fd691242c183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:47.439247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86ae99-9dfa-4262-b603-fd691242c183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:47.440792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86ae99-9dfa-4262-b603-fd691242c183 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:47.454518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 280/556] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:47.459897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313f4fe-f322-48fc-8fb9-be2200d504e4 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.460187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313f4fe-f322-48fc-8fb9-be2200d504e4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.460461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f313f4fe-f322-48fc-8fb9-be2200d504e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.464026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:47.484175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f313f4fe-f322-48fc-8fb9-be2200d504e4 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:47.485753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f313f4fe-f322-48fc-8fb9-be2200d504e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:47.566224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14e421-9264-471e-94be-4639530c91b5 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:47.580514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 281/557] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:47.580514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313f4fe-f322-48fc-8fb9-be2200d504e4 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:47.632128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3285f87-d250-4c76-8b1c-471b34badebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:47.633088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3285f87-d250-4c76-8b1c-471b34badebd None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.633521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3285f87-d250-4c76-8b1c-471b34badebd None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.633926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3285f87-d250-4c76-8b1c-471b34badebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.685473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:47.698990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f313f4fe-f322-48fc-8fb9-be2200d504e4 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:47.738082 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3285f87-d250-4c76-8b1c-471b34badebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:47.740519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3285f87-d250-4c76-8b1c-471b34badebd None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:47.747736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4200ce-a79c-4525-832b-26a0a00beff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:47.748462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4200ce-a79c-4525-832b-26a0a00beff0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.749058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4200ce-a79c-4525-832b-26a0a00beff0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.749448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4200ce-a79c-4525-832b-26a0a00beff0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.757189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4200ce-a79c-4525-832b-26a0a00beff0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:47.758716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4200ce-a79c-4525-832b-26a0a00beff0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:47.759085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-cb4200ce-a79c-4525-832b-26a0a00beff0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:47.771055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:47.773293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 282/558] 10.222.0.105 () {56 vars in 1098 bytes} [Wed Aug 24 04:41:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e722b9f-6949-40b3-8d20-d7bb6bf55d07 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.774468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e722b9f-6949-40b3-8d20-d7bb6bf55d07 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.775208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e722b9f-6949-40b3-8d20-d7bb6bf55d07 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.780582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 283/559] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:47.781441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e722b9f-6949-40b3-8d20-d7bb6bf55d07 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:47.895752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 277/560] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:47.896583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e331-186f-483d-bc59-8d0633e2ad89 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.896828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e331-186f-483d-bc59-8d0633e2ad89 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.897070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3e331-186f-483d-bc59-8d0633e2ad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.905307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e722b9f-6949-40b3-8d20-d7bb6bf55d07 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:47.915621 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-1cc3e331-186f-483d-bc59-8d0633e2ad89 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:47.917019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-1cc3e331-186f-483d-bc59-8d0633e2ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:47.947490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0f6b9-d3d6-4a12-86a0-bd647fa7aca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:47.948098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0f6b9-d3d6-4a12-86a0-bd647fa7aca0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:47.948381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0f6b9-d3d6-4a12-86a0-bd647fa7aca0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:47.948658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0f6b9-d3d6-4a12-86a0-bd647fa7aca0 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:47.949440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0f6b9-d3d6-4a12-86a0-bd647fa7aca0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:47.949754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0f6b9-d3d6-4a12-86a0-bd647fa7aca0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:47.951195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced0f6b9-d3d6-4a12-86a0-bd647fa7aca0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:47.968343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 284/561] 10.222.0.105 () {56 vars in 1138 bytes} [Wed Aug 24 04:41:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:47.969176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0f6b9-d3d6-4a12-86a0-bd647fa7aca0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:47.980317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:48.003124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 278/562] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:47 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:48.003124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3e331-186f-483d-bc59-8d0633e2ad89 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:48.056473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:48.108904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3e331-186f-483d-bc59-8d0633e2ad89 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.154136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed387cd-8d64-463f-a328-8a06fb3a07f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:48.154579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced0f6b9-d3d6-4a12-86a0-bd647fa7aca0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.154766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed387cd-8d64-463f-a328-8a06fb3a07f3 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.155115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed387cd-8d64-463f-a328-8a06fb3a07f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.156109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed387cd-8d64-463f-a328-8a06fb3a07f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.169236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed387cd-8d64-463f-a328-8a06fb3a07f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-7ed387cd-8d64-463f-a328-8a06fb3a07f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:48.175873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:48.176999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:48.177937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 279/563] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:48.178976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed387cd-8d64-463f-a328-8a06fb3a07f3 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:48.209902 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1ad6f-f9a5-4f2b-94cb-29dc4de382ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:48.210635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1ad6f-f9a5-4f2b-94cb-29dc4de382ce None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.211014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1ad6f-f9a5-4f2b-94cb-29dc4de382ce None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.211375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1ad6f-f9a5-4f2b-94cb-29dc4de382ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.223956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1ad6f-f9a5-4f2b-94cb-29dc4de382ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:48.225787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1ad6f-f9a5-4f2b-94cb-29dc4de382ce None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-23d1ad6f-f9a5-4f2b-94cb-29dc4de382ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:48.242937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:48.244126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:48.245239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:48.245239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:48.245239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:48.250331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 285/564] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:48.250331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1ad6f-f9a5-4f2b-94cb-29dc4de382ce None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:48.314842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:48.320469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed387cd-8d64-463f-a328-8a06fb3a07f3 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.327871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:48.356583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856a6f6-e8ae-406f-b8db-186deab016f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:48.357271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856a6f6-e8ae-406f-b8db-186deab016f3 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.357604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856a6f6-e8ae-406f-b8db-186deab016f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.357870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856a6f6-e8ae-406f-b8db-186deab016f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.358106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1ad6f-f9a5-4f2b-94cb-29dc4de382ce None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.368310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856a6f6-e8ae-406f-b8db-186deab016f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:48.369775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856a6f6-e8ae-406f-b8db-186deab016f3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-0856a6f6-e8ae-406f-b8db-186deab016f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:48.385322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:48.386408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:48.387327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:48.387327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:48.387327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:48.387327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 280/565] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:48.387572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856a6f6-e8ae-406f-b8db-186deab016f3 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:48.397191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e78a6-1f12-4025-8a3f-599f81155bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:48.397791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e78a6-1f12-4025-8a3f-599f81155bfd None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.398062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e78a6-1f12-4025-8a3f-599f81155bfd None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.398329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56e78a6-1f12-4025-8a3f-599f81155bfd None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.398992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e78a6-1f12-4025-8a3f-599f81155bfd None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:48.399318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e78a6-1f12-4025-8a3f-599f81155bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:48.400658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56e78a6-1f12-4025-8a3f-599f81155bfd None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:48.408711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 286/566] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:48 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:48.409783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e78a6-1f12-4025-8a3f-599f81155bfd None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:48.490283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856a6f6-e8ae-406f-b8db-186deab016f3 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.508915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56e78a6-1f12-4025-8a3f-599f81155bfd None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.523910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520cb856-a69f-465c-8092-d95c98a0e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:48.524513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cb856-a69f-465c-8092-d95c98a0e25c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.524793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cb856-a69f-465c-8092-d95c98a0e25c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.525066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cb856-a69f-465c-8092-d95c98a0e25c None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.525699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520cb856-a69f-465c-8092-d95c98a0e25c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:48.526006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520cb856-a69f-465c-8092-d95c98a0e25c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:48.527382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520cb856-a69f-465c-8092-d95c98a0e25c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:48.531379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:48.538899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 281/567] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:48.540101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855ff5b-cfa2-4cf9-ba11-2e5e6deb44dc None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.540389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855ff5b-cfa2-4cf9-ba11-2e5e6deb44dc None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.540665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855ff5b-cfa2-4cf9-ba11-2e5e6deb44dc None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.541996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 282/568] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:48 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:48.542542 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855ff5b-cfa2-4cf9-ba11-2e5e6deb44dc None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:48.550505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0195dd-e51b-42c0-90ae-e1a0a85235ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:48.551181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0195dd-e51b-42c0-90ae-e1a0a85235ac None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.551464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0195dd-e51b-42c0-90ae-e1a0a85235ac None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.551734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0195dd-e51b-42c0-90ae-e1a0a85235ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.561271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:48.563222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0195dd-e51b-42c0-90ae-e1a0a85235ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-ec0195dd-e51b-42c0-90ae-e1a0a85235ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:48.568371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:48.569733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 287/569] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:48.570748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0195dd-e51b-42c0-90ae-e1a0a85235ac None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:48.649351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855ff5b-cfa2-4cf9-ba11-2e5e6deb44dc None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.670387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0195dd-e51b-42c0-90ae-e1a0a85235ac None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.683218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e4a54-95fa-4a83-91c0-fe5ebff68b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:48.683815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e4a54-95fa-4a83-91c0-fe5ebff68b31 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.684088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e4a54-95fa-4a83-91c0-fe5ebff68b31 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.684356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765e4a54-95fa-4a83-91c0-fe5ebff68b31 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.684979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e4a54-95fa-4a83-91c0-fe5ebff68b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:48.685273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e4a54-95fa-4a83-91c0-fe5ebff68b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:48.686629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765e4a54-95fa-4a83-91c0-fe5ebff68b31 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:48.698338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 283/570] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:48.699141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e4a54-95fa-4a83-91c0-fe5ebff68b31 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:48.709249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec77c2c-045e-43d6-b8a8-9bc1e7a9c403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:48.710233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec77c2c-045e-43d6-b8a8-9bc1e7a9c403 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.710514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec77c2c-045e-43d6-b8a8-9bc1e7a9c403 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.710786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec77c2c-045e-43d6-b8a8-9bc1e7a9c403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.722206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec77c2c-045e-43d6-b8a8-9bc1e7a9c403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:48.723677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec77c2c-045e-43d6-b8a8-9bc1e7a9c403 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-7ec77c2c-045e-43d6-b8a8-9bc1e7a9c403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:48.737506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:48.738499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:48.739434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:48.739434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:48.739434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:48.739434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 288/571] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:48.739794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9e05b-b33d-419b-97a6-7c2537704773 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.740053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9e05b-b33d-419b-97a6-7c2537704773 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.740265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9e05b-b33d-419b-97a6-7c2537704773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.757319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-73e9e05b-b33d-419b-97a6-7c2537704773 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:48.758527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-73e9e05b-b33d-419b-97a6-7c2537704773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:48.778549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:48.785909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:48.800940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765e4a54-95fa-4a83-91c0-fe5ebff68b31 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.823826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 289/572] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:48 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:48.824576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9e05b-b33d-419b-97a6-7c2537704773 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:48.834051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67321a-5b6b-4e34-b4f5-11eecac83576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:48.834672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67321a-5b6b-4e34-b4f5-11eecac83576 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.834944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67321a-5b6b-4e34-b4f5-11eecac83576 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.836495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67321a-5b6b-4e34-b4f5-11eecac83576 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.841920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67321a-5b6b-4e34-b4f5-11eecac83576 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:48.843936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc67321a-5b6b-4e34-b4f5-11eecac83576 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-dc67321a-5b6b-4e34-b4f5-11eecac83576 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:48.855344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:48.856323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:48.857199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:48.857199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:48.857199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:48.857199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:48.857199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:48.857199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:48.857199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:48.857199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:48.857199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 284/573] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:48.857199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67321a-5b6b-4e34-b4f5-11eecac83576 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:48.942100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9e05b-b33d-419b-97a6-7c2537704773 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.960344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67321a-5b6b-4e34-b4f5-11eecac83576 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:48.978122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa39b83-7595-4062-bb9b-3fbee4d3ec47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:48.978727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa39b83-7595-4062-bb9b-3fbee4d3ec47 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:48.979003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa39b83-7595-4062-bb9b-3fbee4d3ec47 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:48.979303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa39b83-7595-4062-bb9b-3fbee4d3ec47 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:48.979959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa39b83-7595-4062-bb9b-3fbee4d3ec47 None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:48.980258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa39b83-7595-4062-bb9b-3fbee4d3ec47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:48.981620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa39b83-7595-4062-bb9b-3fbee4d3ec47 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:48.989645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 290/574] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:48 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:48.990424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa39b83-7595-4062-bb9b-3fbee4d3ec47 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.001930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79572f-3273-4218-a9e7-070549fdc641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.002785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79572f-3273-4218-a9e7-070549fdc641 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.003237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79572f-3273-4218-a9e7-070549fdc641 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.003651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf79572f-3273-4218-a9e7-070549fdc641 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.006496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:49.011412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79572f-3273-4218-a9e7-070549fdc641 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.012840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf79572f-3273-4218-a9e7-070549fdc641 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:49.020933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-bf79572f-3273-4218-a9e7-070549fdc641 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:49.025016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:49.025890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:49.026770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:49.026770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:49.026770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:49.026770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:49.026770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:49.026770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:49.026770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:49.026770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:49.026770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 285/575] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:49.027755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79572f-3273-4218-a9e7-070549fdc641 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.103245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa39b83-7595-4062-bb9b-3fbee4d3ec47 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.127275 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf79572f-3273-4218-a9e7-070549fdc641 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.136890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36136f-36c7-4aa3-b652-2d7769032d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.137494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36136f-36c7-4aa3-b652-2d7769032d6d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.137772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36136f-36c7-4aa3-b652-2d7769032d6d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.138043 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce36136f-36c7-4aa3-b652-2d7769032d6d None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.138677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36136f-36c7-4aa3-b652-2d7769032d6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:49.138975 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36136f-36c7-4aa3-b652-2d7769032d6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.141006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce36136f-36c7-4aa3-b652-2d7769032d6d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:49.152994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 291/576] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:49.153801 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36136f-36c7-4aa3-b652-2d7769032d6d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.164093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d44e7d-613d-4c37-8a83-1a2272267349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.164699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d44e7d-613d-4c37-8a83-1a2272267349 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.164976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d44e7d-613d-4c37-8a83-1a2272267349 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.165246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d44e7d-613d-4c37-8a83-1a2272267349 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.165929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d44e7d-613d-4c37-8a83-1a2272267349 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:49.166232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d44e7d-613d-4c37-8a83-1a2272267349 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.167675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d44e7d-613d-4c37-8a83-1a2272267349 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:49.179553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 286/577] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:49.180368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d44e7d-613d-4c37-8a83-1a2272267349 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.237529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:49.242343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:49.257172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce36136f-36c7-4aa3-b652-2d7769032d6d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.280751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d44e7d-613d-4c37-8a83-1a2272267349 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.291768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81f886-d8fd-46aa-aa03-ca0dcbacd00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.292386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81f886-d8fd-46aa-aa03-ca0dcbacd00e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.292661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81f886-d8fd-46aa-aa03-ca0dcbacd00e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.292926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81f886-d8fd-46aa-aa03-ca0dcbacd00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.303819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81f886-d8fd-46aa-aa03-ca0dcbacd00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.305252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81f886-d8fd-46aa-aa03-ca0dcbacd00e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:49.314496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a1bf4-963d-4f2f-9a64-cf3356758d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.315128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1bf4-963d-4f2f-9a64-cf3356758d8a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.315412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1bf4-963d-4f2f-9a64-cf3356758d8a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.315686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a1bf4-963d-4f2f-9a64-cf3356758d8a None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.316371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a1bf4-963d-4f2f-9a64-cf3356758d8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:49.316675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a1bf4-963d-4f2f-9a64-cf3356758d8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.318084 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a1bf4-963d-4f2f-9a64-cf3356758d8a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-ae81f886-d8fd-46aa-aa03-ca0dcbacd00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:49.320438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:49.321367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:49.322305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:49.322305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:49.322305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:49.322305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 292/578] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:49.322305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81f886-d8fd-46aa-aa03-ca0dcbacd00e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.329846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 287/579] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:49.330650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a1bf4-963d-4f2f-9a64-cf3356758d8a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.422729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81f886-d8fd-46aa-aa03-ca0dcbacd00e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.426943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a1bf4-963d-4f2f-9a64-cf3356758d8a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.442659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:49.456551 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fca78-ba1d-4cda-81d0-5198d219ae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.457166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fca78-ba1d-4cda-81d0-5198d219ae4b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.457442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fca78-ba1d-4cda-81d0-5198d219ae4b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.457708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70fca78-ba1d-4cda-81d0-5198d219ae4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.460083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8106e-e161-4242-a080-4a9c245dd67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.460725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8106e-e161-4242-a080-4a9c245dd67c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.461001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8106e-e161-4242-a080-4a9c245dd67c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.461272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8106e-e161-4242-a080-4a9c245dd67c None admin] PATH_INFO: `/v3/users/glance` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.464309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:49.468669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70fca78-ba1d-4cda-81d0-5198d219ae4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:49.469126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8106e-e161-4242-a080-4a9c245dd67c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.470584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8106e-e161-4242-a080-4a9c245dd67c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-a70fca78-ba1d-4cda-81d0-5198d219ae4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:49.473458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:49.474626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 293/580] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:49.475736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fca78-ba1d-4cda-81d0-5198d219ae4b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-d4c8106e-e161-4242-a080-4a9c245dd67c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:49.486040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:49.487340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 288/581] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:49.488534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8106e-e161-4242-a080-4a9c245dd67c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.582263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70fca78-ba1d-4cda-81d0-5198d219ae4b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.585823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8106e-e161-4242-a080-4a9c245dd67c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.615386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cad6e-aa90-4a85-946e-70fe470740e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.615984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cad6e-aa90-4a85-946e-70fe470740e0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.616262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cad6e-aa90-4a85-946e-70fe470740e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.616533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094cad6e-aa90-4a85-946e-70fe470740e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.617248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cad6e-aa90-4a85-946e-70fe470740e0 None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=e6a894e4e6f24febbd8397fb8c4c7fb4) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:49.617553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cad6e-aa90-4a85-946e-70fe470740e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.618597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95defc08-49eb-4ef1-bbe4-7d5c72c544e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.618991 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094cad6e-aa90-4a85-946e-70fe470740e0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:49.619212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95defc08-49eb-4ef1-bbe4-7d5c72c544e5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.619486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95defc08-49eb-4ef1-bbe4-7d5c72c544e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.619750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95defc08-49eb-4ef1-bbe4-7d5c72c544e5 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.620376 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95defc08-49eb-4ef1-bbe4-7d5c72c544e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:49.620678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95defc08-49eb-4ef1-bbe4-7d5c72c544e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.622030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95defc08-49eb-4ef1-bbe4-7d5c72c544e5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:49.624323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 294/582] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:49 2022] GET /identity/v3/role_assignments?user.id=e6a894e4e6f24febbd8397fb8c4c7fb4&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 260 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:49.625624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cad6e-aa90-4a85-946e-70fe470740e0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.651309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 289/583] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:49.651309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95defc08-49eb-4ef1-bbe4-7d5c72c544e5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.691668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:49.696036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:49.739461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094cad6e-aa90-4a85-946e-70fe470740e0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.763922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95defc08-49eb-4ef1-bbe4-7d5c72c544e5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.777712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2196af-876a-45ee-ac08-1e268d88906c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.778624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2196af-876a-45ee-ac08-1e268d88906c None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.778904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2196af-876a-45ee-ac08-1e268d88906c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.779213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2196af-876a-45ee-ac08-1e268d88906c None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/3c822c8f23b74bae84769046fdb85db4/roles/4f58cf3fe09f44b1b59e224e7077ce04` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.795971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2196af-876a-45ee-ac08-1e268d88906c None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=3c822c8f23b74bae84769046fdb85db4, role_id=4f58cf3fe09f44b1b59e224e7077ce04)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.798448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2196af-876a-45ee-ac08-1e268d88906c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:49.802344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d24bf7-2a57-41f6-bfbe-26abb4fe92c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.802949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d24bf7-2a57-41f6-bfbe-26abb4fe92c8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.803252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d24bf7-2a57-41f6-bfbe-26abb4fe92c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.803519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d24bf7-2a57-41f6-bfbe-26abb4fe92c8 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.804170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d24bf7-2a57-41f6-bfbe-26abb4fe92c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:49.804470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d24bf7-2a57-41f6-bfbe-26abb4fe92c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.805801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d24bf7-2a57-41f6-bfbe-26abb4fe92c8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:49.814016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 290/584] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:49 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:49.818723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d24bf7-2a57-41f6-bfbe-26abb4fe92c8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.866986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 295/585] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:41:49 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/3c822c8f23b74bae84769046fdb85db4/roles/4f58cf3fe09f44b1b59e224e7077ce04 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:49.880082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2196af-876a-45ee-ac08-1e268d88906c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:49.919630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:49.929656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d24bf7-2a57-41f6-bfbe-26abb4fe92c8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:49.972511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161fffe-9bda-4690-af89-da7acf3f7d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:49.973623 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161fffe-9bda-4690-af89-da7acf3f7d35 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:49.974184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161fffe-9bda-4690-af89-da7acf3f7d35 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:49.974656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161fffe-9bda-4690-af89-da7acf3f7d35 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:49.975700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161fffe-9bda-4690-af89-da7acf3f7d35 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:49.976240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161fffe-9bda-4690-af89-da7acf3f7d35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:49.978165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0161fffe-9bda-4690-af89-da7acf3f7d35 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.002172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 291/586] 10.222.0.105 () {56 vars in 1142 bytes} [Wed Aug 24 04:41:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:50.003525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161fffe-9bda-4690-af89-da7acf3f7d35 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:50.042772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2196af-876a-45ee-ac08-1e268d88906c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:50.062531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:50.088173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a03ae-b131-4e09-bbae-2956aa149448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:50.088810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a03ae-b131-4e09-bbae-2956aa149448 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:50.089090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a03ae-b131-4e09-bbae-2956aa149448 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:50.089360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a03ae-b131-4e09-bbae-2956aa149448 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:50.098215 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a03ae-b131-4e09-bbae-2956aa149448 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:50.099697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a03ae-b131-4e09-bbae-2956aa149448 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-108a03ae-b131-4e09-bbae-2956aa149448 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:50.112170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:50.113476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:50.114418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 296/587] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:50.115010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a03ae-b131-4e09-bbae-2956aa149448 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:50.139333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161fffe-9bda-4690-af89-da7acf3f7d35 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:50.183046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3f891-80d8-4e82-aea1-13481c25d989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:50.183046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f891-80d8-4e82-aea1-13481c25d989 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:50.183046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f891-80d8-4e82-aea1-13481c25d989 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:50.183046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f891-80d8-4e82-aea1-13481c25d989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:50.190258 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3f891-80d8-4e82-aea1-13481c25d989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:50.190258 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3f891-80d8-4e82-aea1-13481c25d989 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-feb3f891-80d8-4e82-aea1-13481c25d989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:50.206112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:50.207140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:50.208057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:50.208057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:50.208057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:50.208057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 292/588] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:50.208703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3f891-80d8-4e82-aea1-13481c25d989 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:50.247486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:50.249451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a03ae-b131-4e09-bbae-2956aa149448 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:50.293454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502653fe-2adc-4b2c-a8c8-8dd757ef47d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:50.294137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502653fe-2adc-4b2c-a8c8-8dd757ef47d4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:50.294728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502653fe-2adc-4b2c-a8c8-8dd757ef47d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:50.295008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502653fe-2adc-4b2c-a8c8-8dd757ef47d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:50.310080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502653fe-2adc-4b2c-a8c8-8dd757ef47d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:50.311835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502653fe-2adc-4b2c-a8c8-8dd757ef47d4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-502653fe-2adc-4b2c-a8c8-8dd757ef47d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:50.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:50.335114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:50.337233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:50.337233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:50.337233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:50.337616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3f891-80d8-4e82-aea1-13481c25d989 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:50.343052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 297/589] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:50.343052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502653fe-2adc-4b2c-a8c8-8dd757ef47d4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:50.374024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93c663-d102-44ba-9f85-13c05a9d777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:50.374630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c663-d102-44ba-9f85-13c05a9d777a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:50.374934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c663-d102-44ba-9f85-13c05a9d777a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:50.375232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93c663-d102-44ba-9f85-13c05a9d777a None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:50.375961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93c663-d102-44ba-9f85-13c05a9d777a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:50.376261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93c663-d102-44ba-9f85-13c05a9d777a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:50.376641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:50.377734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93c663-d102-44ba-9f85-13c05a9d777a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.397126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 293/590] 10.222.0.105 () {56 vars in 1142 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:50.397967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93c663-d102-44ba-9f85-13c05a9d777a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:50.475635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502653fe-2adc-4b2c-a8c8-8dd757ef47d4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:50.483936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:50.520953 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93c663-d102-44ba-9f85-13c05a9d777a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:50.523694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34c494-52cd-4ed1-a8cf-f976705c0d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:50.525381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34c494-52cd-4ed1-a8cf-f976705c0d40 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:50.525813 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34c494-52cd-4ed1-a8cf-f976705c0d40 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:50.526235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34c494-52cd-4ed1-a8cf-f976705c0d40 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:50.527039 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34c494-52cd-4ed1-a8cf-f976705c0d40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:50.527484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34c494-52cd-4ed1-a8cf-f976705c0d40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:50.529018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34c494-52cd-4ed1-a8cf-f976705c0d40 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.550027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 298/591] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:50.550027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34c494-52cd-4ed1-a8cf-f976705c0d40 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:50.563291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341703a5-b3a0-47f1-9810-43f64da10731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:50.563958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341703a5-b3a0-47f1-9810-43f64da10731 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:50.564234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341703a5-b3a0-47f1-9810-43f64da10731 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:50.564504 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341703a5-b3a0-47f1-9810-43f64da10731 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:50.565192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341703a5-b3a0-47f1-9810-43f64da10731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:50.565494 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341703a5-b3a0-47f1-9810-43f64da10731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:50.566847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341703a5-b3a0-47f1-9810-43f64da10731 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.584202 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 294/592] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:50.584202 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341703a5-b3a0-47f1-9810-43f64da10731 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:50.664713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:50.684500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34c494-52cd-4ed1-a8cf-f976705c0d40 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:50.698591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341703a5-b3a0-47f1-9810-43f64da10731 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:50.729010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:50.732606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602fc22-a19a-4adc-8be8-8f5606fbcdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:50.733233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602fc22-a19a-4adc-8be8-8f5606fbcdd0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:50.733508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602fc22-a19a-4adc-8be8-8f5606fbcdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:50.733775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602fc22-a19a-4adc-8be8-8f5606fbcdd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:50.740832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e6927-9441-488e-9ff5-096551ccd15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:50.741729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e6927-9441-488e-9ff5-096551ccd15c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:50.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e6927-9441-488e-9ff5-096551ccd15c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:50.745151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e6927-9441-488e-9ff5-096551ccd15c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:50.747230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602fc22-a19a-4adc-8be8-8f5606fbcdd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:50.748708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602fc22-a19a-4adc-8be8-8f5606fbcdd0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.753045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e6927-9441-488e-9ff5-096551ccd15c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:50.754857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e6927-9441-488e-9ff5-096551ccd15c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-8602fc22-a19a-4adc-8be8-8f5606fbcdd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:50.767155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:50.768150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:50.769085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:50.769085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:50.769085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-703e6927-9441-488e-9ff5-096551ccd15c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:50.770237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:50.771193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:50.772141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:50.772141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:50.772141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:50.772141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:50.772141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:50.772141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:50.772141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:50.772141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:50.772141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 04:41:50.772141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:50.773332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 299/593] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:50.774183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602fc22-a19a-4adc-8be8-8f5606fbcdd0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:50.779994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 295/594] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:50.781726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e6927-9441-488e-9ff5-096551ccd15c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:50.894102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602fc22-a19a-4adc-8be8-8f5606fbcdd0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:50.920205 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e6927-9441-488e-9ff5-096551ccd15c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:50.936110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2b056-bce3-4dd0-8ce2-d07204f2aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:50.937050 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2b056-bce3-4dd0-8ce2-d07204f2aa69 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:50.938589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2b056-bce3-4dd0-8ce2-d07204f2aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:50.938872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2b056-bce3-4dd0-8ce2-d07204f2aa69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:50.942645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:50.948972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2b056-bce3-4dd0-8ce2-d07204f2aa69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:50.951337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2b056-bce3-4dd0-8ce2-d07204f2aa69 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.961716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90ca81-f5d0-4d86-970a-0b65d7e26ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:50.962403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90ca81-f5d0-4d86-970a-0b65d7e26ec4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:50.962754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90ca81-f5d0-4d86-970a-0b65d7e26ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:50.963137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90ca81-f5d0-4d86-970a-0b65d7e26ec4 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:50.963873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90ca81-f5d0-4d86-970a-0b65d7e26ec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:50.964246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90ca81-f5d0-4d86-970a-0b65d7e26ec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-81d2b056-bce3-4dd0-8ce2-d07204f2aa69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:50.965440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:50.966311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:50.967412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:50.967412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:50.967412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:50.967412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:50.967412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:50.967412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:50.967412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:50.967412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:50.968287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90ca81-f5d0-4d86-970a-0b65d7e26ec4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:50.968374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 300/595] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:50.968374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2b056-bce3-4dd0-8ce2-d07204f2aa69 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:50.986333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 296/596] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:50.987375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90ca81-f5d0-4d86-970a-0b65d7e26ec4 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:51.054069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:51.093190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2b056-bce3-4dd0-8ce2-d07204f2aa69 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:51.127934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90ca81-f5d0-4d86-970a-0b65d7e26ec4 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:51.135554 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcf40e-0035-4f83-99d0-ef1ed557dcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:51.136611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcf40e-0035-4f83-99d0-ef1ed557dcf0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.136900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcf40e-0035-4f83-99d0-ef1ed557dcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.137178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dcf40e-0035-4f83-99d0-ef1ed557dcf0 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.138264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dcf40e-0035-4f83-99d0-ef1ed557dcf0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:51.138579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dcf40e-0035-4f83-99d0-ef1ed557dcf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:51.140799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dcf40e-0035-4f83-99d0-ef1ed557dcf0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:51.163749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 301/597] 10.222.0.105 () {56 vars in 1144 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:51.164662 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcf40e-0035-4f83-99d0-ef1ed557dcf0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:51.169960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb850b-7238-4157-a521-d07428ea6ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:51.170726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb850b-7238-4157-a521-d07428ea6ab6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.171118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb850b-7238-4157-a521-d07428ea6ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.171464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb850b-7238-4157-a521-d07428ea6ab6 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.172965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb850b-7238-4157-a521-d07428ea6ab6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:51.173341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb850b-7238-4157-a521-d07428ea6ab6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:51.175052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb850b-7238-4157-a521-d07428ea6ab6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:51.194063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 297/598] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:51.195472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb850b-7238-4157-a521-d07428ea6ab6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:51.245739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:51.309695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dcf40e-0035-4f83-99d0-ef1ed557dcf0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:51.319558 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb850b-7238-4157-a521-d07428ea6ab6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:51.330187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:51.347466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd8c5a-859a-4524-bf93-64e41f0cf556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:51.348195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd8c5a-859a-4524-bf93-64e41f0cf556 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.348547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd8c5a-859a-4524-bf93-64e41f0cf556 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.348890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd8c5a-859a-4524-bf93-64e41f0cf556 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.356394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd8c5a-859a-4524-bf93-64e41f0cf556 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:51.357959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd8c5a-859a-4524-bf93-64e41f0cf556 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:51.366475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62951a-8dd7-403d-8001-a2c9563a551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:51.367377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62951a-8dd7-403d-8001-a2c9563a551c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.367860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62951a-8dd7-403d-8001-a2c9563a551c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.368324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62951a-8dd7-403d-8001-a2c9563a551c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-d6dd8c5a-859a-4524-bf93-64e41f0cf556 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:51.370873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:51.371772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:51.372685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:51.372685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:51.372685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:51.372685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:51.372685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:51.372685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:51.372685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:51.372685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:51.377140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 302/599] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:51.378050 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd8c5a-859a-4524-bf93-64e41f0cf556 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:51.385146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62951a-8dd7-403d-8001-a2c9563a551c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:51.386820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62951a-8dd7-403d-8001-a2c9563a551c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-fc62951a-8dd7-403d-8001-a2c9563a551c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:51.408124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:51.409103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:51.410018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:51.410018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:51.410018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:51.412366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 298/600] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:51.413367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62951a-8dd7-403d-8001-a2c9563a551c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:51.519421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd8c5a-859a-4524-bf93-64e41f0cf556 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:51.523650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62951a-8dd7-403d-8001-a2c9563a551c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:51.539458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:51.559907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9779929-0938-4db1-a7e1-5b7c53415da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:51.560732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9779929-0938-4db1-a7e1-5b7c53415da3 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.561158 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9779929-0938-4db1-a7e1-5b7c53415da3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.561593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9779929-0938-4db1-a7e1-5b7c53415da3 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.562436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9779929-0938-4db1-a7e1-5b7c53415da3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:51.563148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9779929-0938-4db1-a7e1-5b7c53415da3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:51.564832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9779929-0938-4db1-a7e1-5b7c53415da3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:51.566212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d669c3-bf00-4396-b745-5a498de70aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:51.566945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d669c3-bf00-4396-b745-5a498de70aa4 None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.567319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d669c3-bf00-4396-b745-5a498de70aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.567664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d669c3-bf00-4396-b745-5a498de70aa4 None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/f1d62a4c07c4402ea5803f6e68fb1d0b/roles/4f58cf3fe09f44b1b59e224e7077ce04` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.580695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 299/601] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:51.581854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9779929-0938-4db1-a7e1-5b7c53415da3 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:51.590202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d669c3-bf00-4396-b745-5a498de70aa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=f1d62a4c07c4402ea5803f6e68fb1d0b, role_id=4f58cf3fe09f44b1b59e224e7077ce04)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:51.591872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d669c3-bf00-4396-b745-5a498de70aa4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:51.604882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:51.663248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 303/602] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:41:51 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/f1d62a4c07c4402ea5803f6e68fb1d0b/roles/4f58cf3fe09f44b1b59e224e7077ce04 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:51.664510 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d669c3-bf00-4396-b745-5a498de70aa4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:51.692512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9779929-0938-4db1-a7e1-5b7c53415da3 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:51.732060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6f879-8363-4cf4-8ad7-9ee16a42ea5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:51.733436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6f879-8363-4cf4-8ad7-9ee16a42ea5d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.733888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6f879-8363-4cf4-8ad7-9ee16a42ea5d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.734357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6f879-8363-4cf4-8ad7-9ee16a42ea5d None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.736303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6f879-8363-4cf4-8ad7-9ee16a42ea5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:51.736837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6f879-8363-4cf4-8ad7-9ee16a42ea5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:51.738545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a6f879-8363-4cf4-8ad7-9ee16a42ea5d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:51.752636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 300/603] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:51.754212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51430677-fcbd-4aee-9c49-193de0b9a4c0 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.754692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51430677-fcbd-4aee-9c49-193de0b9a4c0 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.755190 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51430677-fcbd-4aee-9c49-193de0b9a4c0 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.756951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 301/604] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:51.758253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51430677-fcbd-4aee-9c49-193de0b9a4c0 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:51.786716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d669c3-bf00-4396-b745-5a498de70aa4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:51.827296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8311ad-8737-466a-b8db-e68a20624102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:51.827900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8311ad-8737-466a-b8db-e68a20624102 None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.828268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8311ad-8737-466a-b8db-e68a20624102 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.829249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8311ad-8737-466a-b8db-e68a20624102 None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/29af87e1e05b4880b9ec96865994b635/roles/4f58cf3fe09f44b1b59e224e7077ce04` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.849595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8311ad-8737-466a-b8db-e68a20624102 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=29af87e1e05b4880b9ec96865994b635, role_id=4f58cf3fe09f44b1b59e224e7077ce04)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:51.850947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8311ad-8737-466a-b8db-e68a20624102 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:51.851556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:51.876140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51430677-fcbd-4aee-9c49-193de0b9a4c0 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:51.913575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d2491-680b-47d2-82a9-8b613d67cb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:51.914204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d2491-680b-47d2-82a9-8b613d67cb56 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.914482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d2491-680b-47d2-82a9-8b613d67cb56 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.914751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d2491-680b-47d2-82a9-8b613d67cb56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.921668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d2491-680b-47d2-82a9-8b613d67cb56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:51.923351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d2491-680b-47d2-82a9-8b613d67cb56 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-f00d2491-680b-47d2-82a9-8b613d67cb56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:51.934480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:51.935442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:51.936356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:51.936356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:51.936356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:51.936356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:51.936356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:51.936356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:51.936356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:51.936356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:51.936356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 302/605] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:51.936870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7943b-18c8-4e56-a924-90b9e9478099 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.937353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7943b-18c8-4e56-a924-90b9e9478099 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.937516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7943b-18c8-4e56-a924-90b9e9478099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.956726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8da7943b-18c8-4e56-a924-90b9e9478099 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:51.957972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8da7943b-18c8-4e56-a924-90b9e9478099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:51.960361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 304/606] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:41:51 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/29af87e1e05b4880b9ec96865994b635/roles/4f58cf3fe09f44b1b59e224e7077ce04 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:51.968109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc926-ce37-4f09-ad7d-1937c8841ec5 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:51.968388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc926-ce37-4f09-ad7d-1937c8841ec5 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:51.968657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc926-ce37-4f09-ad7d-1937c8841ec5 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:51.973274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 305/607] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:51 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:51.974886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc926-ce37-4f09-ad7d-1937c8841ec5 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:51.999078 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:52.055376 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 303/608] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:52.057629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b961-2b67-4749-9faf-f14f08b0325b None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:52.058146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b961-2b67-4749-9faf-f14f08b0325b None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:52.058644 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770b961-2b67-4749-9faf-f14f08b0325b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:52.082041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e770b961-2b67-4749-9faf-f14f08b0325b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:52.084755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e770b961-2b67-4749-9faf-f14f08b0325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:52.144433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc926-ce37-4f09-ad7d-1937c8841ec5 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:52.177773 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:52.178345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 304/609] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:52.211252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b06c-fc0a-425d-8cee-a5497a233311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:52.211875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b06c-fc0a-425d-8cee-a5497a233311 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:52.212149 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b06c-fc0a-425d-8cee-a5497a233311 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:52.212465 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68b06c-fc0a-425d-8cee-a5497a233311 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:52.213225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68b06c-fc0a-425d-8cee-a5497a233311 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:52.213463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68b06c-fc0a-425d-8cee-a5497a233311 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:52.214943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f68b06c-fc0a-425d-8cee-a5497a233311 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:52.228812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 306/610] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:52.234437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770b961-2b67-4749-9faf-f14f08b0325b None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:52.374745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e770b961-2b67-4749-9faf-f14f08b0325b None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:52.419325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40156-1f3e-44e9-90d4-264588da5e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:52.420346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40156-1f3e-44e9-90d4-264588da5e9d None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:52.420831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40156-1f3e-44e9-90d4-264588da5e9d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:52.421328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d40156-1f3e-44e9-90d4-264588da5e9d None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/44d2b4015ac74b8b8d0a674419f34cf7/roles/4f58cf3fe09f44b1b59e224e7077ce04` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:52.442495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d40156-1f3e-44e9-90d4-264588da5e9d None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=44d2b4015ac74b8b8d0a674419f34cf7, role_id=4f58cf3fe09f44b1b59e224e7077ce04)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:52.444339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d40156-1f3e-44e9-90d4-264588da5e9d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:52.451548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b06c-fc0a-425d-8cee-a5497a233311 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:52.521424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 305/611] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:41:52 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/44d2b4015ac74b8b8d0a674419f34cf7/roles/4f58cf3fe09f44b1b59e224e7077ce04 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:52.522585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40156-1f3e-44e9-90d4-264588da5e9d None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:52.568687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:52.619884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68b06c-fc0a-425d-8cee-a5497a233311 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:52.622737 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:52.695341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d40156-1f3e-44e9-90d4-264588da5e9d None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:52.720425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fbb9a-70bc-428c-b6e6-481ba9e4cd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:52.721428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fbb9a-70bc-428c-b6e6-481ba9e4cd55 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:52.721541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fbb9a-70bc-428c-b6e6-481ba9e4cd55 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:52.721759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40fbb9a-70bc-428c-b6e6-481ba9e4cd55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:52.734330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40fbb9a-70bc-428c-b6e6-481ba9e4cd55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-d40fbb9a-70bc-428c-b6e6-481ba9e4cd55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:52.741376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:52.742338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:52.743234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 307/612] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:52.749300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fbb9a-70bc-428c-b6e6-481ba9e4cd55 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:52.751196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2e92-8cc9-4adc-b4d7-a32e85af7e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:52.752394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2e92-8cc9-4adc-b4d7-a32e85af7e40 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:52.753001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2e92-8cc9-4adc-b4d7-a32e85af7e40 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:52.753415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2e92-8cc9-4adc-b4d7-a32e85af7e40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:52.770973 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e2e92-8cc9-4adc-b4d7-a32e85af7e40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-471e2e92-8cc9-4adc-b4d7-a32e85af7e40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:52.777150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:52.778272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:52.779358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:52.781518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 306/613] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:52.787268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2e92-8cc9-4adc-b4d7-a32e85af7e40 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:52.870748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40fbb9a-70bc-428c-b6e6-481ba9e4cd55 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:52.902203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2e92-8cc9-4adc-b4d7-a32e85af7e40 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:52.921445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:52.926579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f37dd4-f028-4286-a352-77a4b0209375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:52.927424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f37dd4-f028-4286-a352-77a4b0209375 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:52.931750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f37dd4-f028-4286-a352-77a4b0209375 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:52.931750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f37dd4-f028-4286-a352-77a4b0209375 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:52.931750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f37dd4-f028-4286-a352-77a4b0209375 None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:52.931750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f37dd4-f028-4286-a352-77a4b0209375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:52.931750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f37dd4-f028-4286-a352-77a4b0209375 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:52.932159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:52.940305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 308/614] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:52 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:52.946127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f37dd4-f028-4286-a352-77a4b0209375 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:52.985810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c3ffd-7fdf-4c44-91e1-d3d634f823c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:52.986435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c3ffd-7fdf-4c44-91e1-d3d634f823c4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:52.986716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c3ffd-7fdf-4c44-91e1-d3d634f823c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:52.986992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c3ffd-7fdf-4c44-91e1-d3d634f823c4 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:52.987682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c3ffd-7fdf-4c44-91e1-d3d634f823c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:52.987986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c3ffd-7fdf-4c44-91e1-d3d634f823c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:52.989415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c3ffd-7fdf-4c44-91e1-d3d634f823c4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:52.998826 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 307/615] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:52 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:41:53.004560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c3ffd-7fdf-4c44-91e1-d3d634f823c4 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:53.113213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f37dd4-f028-4286-a352-77a4b0209375 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:53.182729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0903fa-e2cf-467b-936e-29884c9064e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:53.183408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0903fa-e2cf-467b-936e-29884c9064e3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:53.183685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0903fa-e2cf-467b-936e-29884c9064e3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:53.183953 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0903fa-e2cf-467b-936e-29884c9064e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:53.199839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0903fa-e2cf-467b-936e-29884c9064e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:53.201235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0903fa-e2cf-467b-936e-29884c9064e3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:53.214076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c3ffd-7fdf-4c44-91e1-d3d634f823c4 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-7c0903fa-e2cf-467b-936e-29884c9064e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:53.230533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:53.231726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:53.232930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:53.232930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:53.232930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:53.233203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 309/616] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:53.244094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0903fa-e2cf-467b-936e-29884c9064e3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:53.261751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020cddd-4830-420c-aa7c-eadf5b892dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:53.262391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020cddd-4830-420c-aa7c-eadf5b892dce None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:53.262673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020cddd-4830-420c-aa7c-eadf5b892dce None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:53.262945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020cddd-4830-420c-aa7c-eadf5b892dce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:53.274883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020cddd-4830-420c-aa7c-eadf5b892dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:53.276609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020cddd-4830-420c-aa7c-eadf5b892dce None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-a020cddd-4830-420c-aa7c-eadf5b892dce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:53.295209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:53.296236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:53.297118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:53.297118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:53.297118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:53.302433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 308/617] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:53.304932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020cddd-4830-420c-aa7c-eadf5b892dce None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:53.344124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:53.353010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:53.444109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0903fa-e2cf-467b-936e-29884c9064e3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:53.495269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020cddd-4830-420c-aa7c-eadf5b892dce None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:53.515106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf2e99-9e6c-4a6a-8480-8f0f024e84a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:53.516192 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf2e99-9e6c-4a6a-8480-8f0f024e84a1 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:53.517292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf2e99-9e6c-4a6a-8480-8f0f024e84a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:53.517941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf2e99-9e6c-4a6a-8480-8f0f024e84a1 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:53.519336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf2e99-9e6c-4a6a-8480-8f0f024e84a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:53.519955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf2e99-9e6c-4a6a-8480-8f0f024e84a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:53.521489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf2e99-9e6c-4a6a-8480-8f0f024e84a1 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:53.552450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 310/618] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:53.553109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1a8d2-a0be-4886-915e-c53c9e71dc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:53.553725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1a8d2-a0be-4886-915e-c53c9e71dc5b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:53.558361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1a8d2-a0be-4886-915e-c53c9e71dc5b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:53.558653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1a8d2-a0be-4886-915e-c53c9e71dc5b None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:53.559353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1a8d2-a0be-4886-915e-c53c9e71dc5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:53.559664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1a8d2-a0be-4886-915e-c53c9e71dc5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:53.561584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf2e99-9e6c-4a6a-8480-8f0f024e84a1 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:53.562850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d1a8d2-a0be-4886-915e-c53c9e71dc5b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:53.591112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 309/619] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:53.596960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1a8d2-a0be-4886-915e-c53c9e71dc5b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:53.752303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf2e99-9e6c-4a6a-8480-8f0f024e84a1 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:53.768600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:53.777351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:53.794500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843a38d-38a4-4ccb-a0e3-5a8c3038f1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:53.795198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843a38d-38a4-4ccb-a0e3-5a8c3038f1ba None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:53.795481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843a38d-38a4-4ccb-a0e3-5a8c3038f1ba None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:53.795752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843a38d-38a4-4ccb-a0e3-5a8c3038f1ba None admin] PATH_INFO: `/v3/users/swift` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:53.803988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843a38d-38a4-4ccb-a0e3-5a8c3038f1ba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:53.805439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843a38d-38a4-4ccb-a0e3-5a8c3038f1ba None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b843a38d-38a4-4ccb-a0e3-5a8c3038f1ba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:53.822278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:53.825821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:53.826805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 311/620] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:41:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:53.829847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843a38d-38a4-4ccb-a0e3-5a8c3038f1ba None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:53.834550 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1a8d2-a0be-4886-915e-c53c9e71dc5b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:53.900459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95d383-4fa8-4649-8f84-1309a1819041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:53.901105 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95d383-4fa8-4649-8f84-1309a1819041 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:53.901485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95d383-4fa8-4649-8f84-1309a1819041 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:53.901759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db95d383-4fa8-4649-8f84-1309a1819041 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:53.910600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95d383-4fa8-4649-8f84-1309a1819041 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:53.912880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db95d383-4fa8-4649-8f84-1309a1819041 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-db95d383-4fa8-4649-8f84-1309a1819041 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:53.930206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:53.931245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:53.932266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:53.932266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:53.932266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:53.932266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:53.932266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:53.932266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:53.932266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:53.932266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:53.932266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:41:53.932266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:53.939681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 310/621] 10.222.0.105 () {56 vars in 1098 bytes} [Wed Aug 24 04:41:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:53.942490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95d383-4fa8-4649-8f84-1309a1819041 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:54.011303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843a38d-38a4-4ccb-a0e3-5a8c3038f1ba None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:54.048754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258c4e23-1c77-4d85-9d95-bcc953d17055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:54.049361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c4e23-1c77-4d85-9d95-bcc953d17055 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.049639 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c4e23-1c77-4d85-9d95-bcc953d17055 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.049906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c4e23-1c77-4d85-9d95-bcc953d17055 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.050634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c4e23-1c77-4d85-9d95-bcc953d17055 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:54.050938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c4e23-1c77-4d85-9d95-bcc953d17055 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:54.052533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258c4e23-1c77-4d85-9d95-bcc953d17055 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:54.076920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 312/622] 10.222.0.105 () {56 vars in 1140 bytes} [Wed Aug 24 04:41:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:54.078338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fad96-e10d-451a-abe9-cc5f529926d7 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.078625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fad96-e10d-451a-abe9-cc5f529926d7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.078903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fad96-e10d-451a-abe9-cc5f529926d7 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.080427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db95d383-4fa8-4649-8f84-1309a1819041 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:54.081461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 313/623] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:54.081461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fad96-e10d-451a-abe9-cc5f529926d7 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:54.116131 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba93a63b-fca7-46d4-9e1f-9e41added122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:54.116343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:54.116747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93a63b-fca7-46d4-9e1f-9e41added122 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.117024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93a63b-fca7-46d4-9e1f-9e41added122 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.117309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba93a63b-fca7-46d4-9e1f-9e41added122 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.118024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93a63b-fca7-46d4-9e1f-9e41added122 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:54.118328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93a63b-fca7-46d4-9e1f-9e41added122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:54.119781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba93a63b-fca7-46d4-9e1f-9e41added122 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:54.134956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:54.138759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 311/624] 10.222.0.105 () {56 vars in 1138 bytes} [Wed Aug 24 04:41:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:54.140241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc4973-e311-4414-8323-0b29e9c5e5e2 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.140524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc4973-e311-4414-8323-0b29e9c5e5e2 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.140797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc4973-e311-4414-8323-0b29e9c5e5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.163335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5edc4973-e311-4414-8323-0b29e9c5e5e2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:54.164538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5edc4973-e311-4414-8323-0b29e9c5e5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:54.206330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fad96-e10d-451a-abe9-cc5f529926d7 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:54.248191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58a2c4-690f-4ace-bb3d-d81b3a0e48c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:54.249302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58a2c4-690f-4ace-bb3d-d81b3a0e48c3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.249882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58a2c4-690f-4ace-bb3d-d81b3a0e48c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.250379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58a2c4-690f-4ace-bb3d-d81b3a0e48c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.263121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 312/625] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:54.263121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc4973-e311-4414-8323-0b29e9c5e5e2 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:54.274463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58a2c4-690f-4ace-bb3d-d81b3a0e48c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:54.276389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e58a2c4-690f-4ace-bb3d-d81b3a0e48c3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-0e58a2c4-690f-4ace-bb3d-d81b3a0e48c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:54.304053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:54.305057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:54.305967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:54.305967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:54.305967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:54.308079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 314/626] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:54.314074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b56698-ae3b-461d-90be-8d73ddbb0ea6 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.314771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b56698-ae3b-461d-90be-8d73ddbb0ea6 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.315424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b56698-ae3b-461d-90be-8d73ddbb0ea6 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.317395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 315/627] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:54 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:54.319208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b56698-ae3b-461d-90be-8d73ddbb0ea6 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:54.382271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc4973-e311-4414-8323-0b29e9c5e5e2 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:54.412098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:54.422009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:54.429705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dfc32-6544-42c0-aae6-91ea99bb571b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:54.430280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dfc32-6544-42c0-aae6-91ea99bb571b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.430583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dfc32-6544-42c0-aae6-91ea99bb571b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.430854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1dfc32-6544-42c0-aae6-91ea99bb571b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.440604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b56698-ae3b-461d-90be-8d73ddbb0ea6 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:54.486508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6becef3f-8601-4079-ab43-f705eefd2510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:54.487152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becef3f-8601-4079-ab43-f705eefd2510 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.487434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becef3f-8601-4079-ab43-f705eefd2510 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.487707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becef3f-8601-4079-ab43-f705eefd2510 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.488792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6becef3f-8601-4079-ab43-f705eefd2510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:54.489155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6becef3f-8601-4079-ab43-f705eefd2510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:54.490527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6becef3f-8601-4079-ab43-f705eefd2510 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:54.511125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 316/628] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:54.511125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f6a7c-bae0-4473-b483-b7594060b9cf None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.511125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f6a7c-bae0-4473-b483-b7594060b9cf None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.511125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f6a7c-bae0-4473-b483-b7594060b9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.523182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dfc32-6544-42c0-aae6-91ea99bb571b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:54.524868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1dfc32-6544-42c0-aae6-91ea99bb571b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:54.530599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-033f6a7c-bae0-4473-b483-b7594060b9cf None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:54.531920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-033f6a7c-bae0-4473-b483-b7594060b9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:54.614996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 317/629] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:54.615809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f6a7c-bae0-4473-b483-b7594060b9cf None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:54.648628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 313/630] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:54 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:54.648628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dfc32-6544-42c0-aae6-91ea99bb571b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:54.684809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:54.708371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:54.742806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033f6a7c-bae0-4473-b483-b7594060b9cf None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:54.761734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1dfc32-6544-42c0-aae6-91ea99bb571b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:54.779150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4fe3a-c683-40d2-b6fc-14e9552302fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:54.779715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4fe3a-c683-40d2-b6fc-14e9552302fa None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.780262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4fe3a-c683-40d2-b6fc-14e9552302fa None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.780535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4fe3a-c683-40d2-b6fc-14e9552302fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.788536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4fe3a-c683-40d2-b6fc-14e9552302fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:54.790369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d4fe3a-c683-40d2-b6fc-14e9552302fa None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:54.796357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa56ac7-913d-45bb-91da-9ef5fc7e50d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:54.797008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa56ac7-913d-45bb-91da-9ef5fc7e50d8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.797255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa56ac7-913d-45bb-91da-9ef5fc7e50d8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.797527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa56ac7-913d-45bb-91da-9ef5fc7e50d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-56d4fe3a-c683-40d2-b6fc-14e9552302fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:54.804795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:54.805874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:54.806946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:54.806946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:54.806946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:54.806946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:54.806946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:54.806946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:54.806946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:54.806946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:54.806946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 318/631] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:54.806946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4fe3a-c683-40d2-b6fc-14e9552302fa None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:54.810159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa56ac7-913d-45bb-91da-9ef5fc7e50d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-ffa56ac7-913d-45bb-91da-9ef5fc7e50d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:54.815170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:54.816126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:54.816962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:54.821001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 314/632] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:54.822566 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2994ca-5980-4d0d-a492-e09ec986b728 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.822855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2994ca-5980-4d0d-a492-e09ec986b728 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.823153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2994ca-5980-4d0d-a492-e09ec986b728 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.824523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 315/633] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:54.825151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2994ca-5980-4d0d-a492-e09ec986b728 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:54.918848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:54.920435 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4fe3a-c683-40d2-b6fc-14e9552302fa None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:54.929377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2994ca-5980-4d0d-a492-e09ec986b728 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:54.947344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:54.955796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e74b9b-cf6a-41e6-a3ed-6da78360c038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:54.956424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e74b9b-cf6a-41e6-a3ed-6da78360c038 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.956704 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e74b9b-cf6a-41e6-a3ed-6da78360c038 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.957002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e74b9b-cf6a-41e6-a3ed-6da78360c038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.964258 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a6c93-8341-4e17-84b1-989f78d0bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:54.964891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a6c93-8341-4e17-84b1-989f78d0bde6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.965167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a6c93-8341-4e17-84b1-989f78d0bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.965437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a6c93-8341-4e17-84b1-989f78d0bde6 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:54.966125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a6c93-8341-4e17-84b1-989f78d0bde6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:54.966427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a6c93-8341-4e17-84b1-989f78d0bde6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:54.967209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e74b9b-cf6a-41e6-a3ed-6da78360c038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:54.967877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a6c93-8341-4e17-84b1-989f78d0bde6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:54.968686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e74b9b-cf6a-41e6-a3ed-6da78360c038 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:54.979064 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 316/634] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:54.979854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a6c93-8341-4e17-84b1-989f78d0bde6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-d8e74b9b-cf6a-41e6-a3ed-6da78360c038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:54.983007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:54.984374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:54.985433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:54.985433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:54.985433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:54.985433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 319/635] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:54.985433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc166e60-4e12-4217-8673-2e2692efa006 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:54.985794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc166e60-4e12-4217-8673-2e2692efa006 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:54.985794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc166e60-4e12-4217-8673-2e2692efa006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.003331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-fc166e60-4e12-4217-8673-2e2692efa006 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:55.004517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fc166e60-4e12-4217-8673-2e2692efa006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:55.070357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 320/636] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:55.071217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc166e60-4e12-4217-8673-2e2692efa006 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.095947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a6c93-8341-4e17-84b1-989f78d0bde6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:55.133378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5accf1-919e-4229-9451-57e371b51c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:55.133980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5accf1-919e-4229-9451-57e371b51c23 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:55.134255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5accf1-919e-4229-9451-57e371b51c23 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:55.134638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5accf1-919e-4229-9451-57e371b51c23 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.135219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5accf1-919e-4229-9451-57e371b51c23 None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:55.135520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5accf1-919e-4229-9451-57e371b51c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:55.136855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5accf1-919e-4229-9451-57e371b51c23 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:55.149996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 317/637] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:54 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:55.149996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5accf1-919e-4229-9451-57e371b51c23 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.164382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:55.177931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc166e60-4e12-4217-8673-2e2692efa006 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:55.192407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:55.218472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af837055-97bb-4e8f-9d10-5260e70f607c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:55.219374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af837055-97bb-4e8f-9d10-5260e70f607c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:55.219647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af837055-97bb-4e8f-9d10-5260e70f607c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:55.219945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af837055-97bb-4e8f-9d10-5260e70f607c None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.232207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af837055-97bb-4e8f-9d10-5260e70f607c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-af837055-97bb-4e8f-9d10-5260e70f607c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:55.237424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:55.238345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:55.239218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 321/638] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:55.240527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af837055-97bb-4e8f-9d10-5260e70f607c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.261368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5accf1-919e-4229-9451-57e371b51c23 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:55.299447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3c566-f7e7-48cd-93f5-b664a56f7244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:55.300036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3c566-f7e7-48cd-93f5-b664a56f7244 None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:55.300320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3c566-f7e7-48cd-93f5-b664a56f7244 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:55.300596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3c566-f7e7-48cd-93f5-b664a56f7244 None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/e6a894e4e6f24febbd8397fb8c4c7fb4/roles/4f58cf3fe09f44b1b59e224e7077ce04` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.316437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3c566-f7e7-48cd-93f5-b664a56f7244 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=e6a894e4e6f24febbd8397fb8c4c7fb4, role_id=4f58cf3fe09f44b1b59e224e7077ce04)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:55.317877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3c566-f7e7-48cd-93f5-b664a56f7244 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:55.345130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af837055-97bb-4e8f-9d10-5260e70f607c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:55.379406 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7ba52-b972-4370-a12d-48cd722db730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:55.379989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7ba52-b972-4370-a12d-48cd722db730 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:55.380268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7ba52-b972-4370-a12d-48cd722db730 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:55.380537 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7ba52-b972-4370-a12d-48cd722db730 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.381165 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7ba52-b972-4370-a12d-48cd722db730 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:55.381461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7ba52-b972-4370-a12d-48cd722db730 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:55.382864 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7ba52-b972-4370-a12d-48cd722db730 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:55.393744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 322/639] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:55.394524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7ba52-b972-4370-a12d-48cd722db730 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.396581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 318/640] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:41:55 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/e6a894e4e6f24febbd8397fb8c4c7fb4/roles/4f58cf3fe09f44b1b59e224e7077ce04 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:41:55.408383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3c566-f7e7-48cd-93f5-b664a56f7244 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.436781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:55.488686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:55.518397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7ba52-b972-4370-a12d-48cd722db730 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:55.531150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3c566-f7e7-48cd-93f5-b664a56f7244 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:55.560800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a23cec-b420-44f1-8f10-dc2c3e599d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:55.561749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a23cec-b420-44f1-8f10-dc2c3e599d8c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:55.562027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a23cec-b420-44f1-8f10-dc2c3e599d8c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:55.562444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a23cec-b420-44f1-8f10-dc2c3e599d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.571141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd448f0-6851-465e-9873-e44a945cb3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:55.571997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd448f0-6851-465e-9873-e44a945cb3d5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:55.572287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd448f0-6851-465e-9873-e44a945cb3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:55.572533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd448f0-6851-465e-9873-e44a945cb3d5 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.573187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd448f0-6851-465e-9873-e44a945cb3d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:55.573481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd448f0-6851-465e-9873-e44a945cb3d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:55.574831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd448f0-6851-465e-9873-e44a945cb3d5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:55.575172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a23cec-b420-44f1-8f10-dc2c3e599d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:55.577295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a23cec-b420-44f1-8f10-dc2c3e599d8c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:55.583991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 319/641] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:55 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:55.584812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd448f0-6851-465e-9873-e44a945cb3d5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-53a23cec-b420-44f1-8f10-dc2c3e599d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:55.596993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:55.598153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:55.599268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:55.599268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:55.599268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:55.600405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 323/642] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:55.601276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a23cec-b420-44f1-8f10-dc2c3e599d8c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.687007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:55.701517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd448f0-6851-465e-9873-e44a945cb3d5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:55.706931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a23cec-b420-44f1-8f10-dc2c3e599d8c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:55.727823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:55.740472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95e4b3-9d1f-4294-9fb2-e7d1242aae19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:55.741162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95e4b3-9d1f-4294-9fb2-e7d1242aae19 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:55.741438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95e4b3-9d1f-4294-9fb2-e7d1242aae19 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:55.741704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95e4b3-9d1f-4294-9fb2-e7d1242aae19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.743402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90a3dd-c06c-4ee6-afe1-faa9014f8a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:55.744028 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90a3dd-c06c-4ee6-afe1-faa9014f8a1b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:55.744305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90a3dd-c06c-4ee6-afe1-faa9014f8a1b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:55.744573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90a3dd-c06c-4ee6-afe1-faa9014f8a1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.751379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90a3dd-c06c-4ee6-afe1-faa9014f8a1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:55.752655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea95e4b3-9d1f-4294-9fb2-e7d1242aae19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:55.753070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90a3dd-c06c-4ee6-afe1-faa9014f8a1b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-ea95e4b3-9d1f-4294-9fb2-e7d1242aae19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:55.757303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:55.758294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:55.759217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 320/643] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:55.759938 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95e4b3-9d1f-4294-9fb2-e7d1242aae19 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-bb90a3dd-c06c-4ee6-afe1-faa9014f8a1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:55.770417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:55.771388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:55.772353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:55.772353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:55.772353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:55.772353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:55.772353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:55.772353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:55.772353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:55.772353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:55.772353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 324/644] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:55.773368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90a3dd-c06c-4ee6-afe1-faa9014f8a1b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.879866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95e4b3-9d1f-4294-9fb2-e7d1242aae19 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:55.884169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90a3dd-c06c-4ee6-afe1-faa9014f8a1b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:55.917533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50378083-48af-4a6b-9370-085d6299cb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:55.918409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50378083-48af-4a6b-9370-085d6299cb0c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:55.918684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50378083-48af-4a6b-9370-085d6299cb0c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:55.918955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50378083-48af-4a6b-9370-085d6299cb0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.922234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d5a5a-1641-4bfc-adfc-0c12e59eade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:55.922850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d5a5a-1641-4bfc-adfc-0c12e59eade0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:55.923158 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d5a5a-1641-4bfc-adfc-0c12e59eade0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:55.923578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d5a5a-1641-4bfc-adfc-0c12e59eade0 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:55.924896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d5a5a-1641-4bfc-adfc-0c12e59eade0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:55.925203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d5a5a-1641-4bfc-adfc-0c12e59eade0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:55.926558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d5a5a-1641-4bfc-adfc-0c12e59eade0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:55.930823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50378083-48af-4a6b-9370-085d6299cb0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:55.932268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50378083-48af-4a6b-9370-085d6299cb0c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:55.938651 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 325/645] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:55.940231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d5a5a-1641-4bfc-adfc-0c12e59eade0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.942982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-50378083-48af-4a6b-9370-085d6299cb0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:55.951877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:55.952892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:55.953788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:55.953788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:55.953788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:55.955835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 321/646] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:55.957274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50378083-48af-4a6b-9370-085d6299cb0c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:55.997167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:56.049830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d5a5a-1641-4bfc-adfc-0c12e59eade0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.062355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50378083-48af-4a6b-9370-085d6299cb0c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.087275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2697e-ae85-4eab-9933-177717718cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:56.087883 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2697e-ae85-4eab-9933-177717718cb2 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:56.088164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2697e-ae85-4eab-9933-177717718cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:56.088434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2697e-ae85-4eab-9933-177717718cb2 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:56.089091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2697e-ae85-4eab-9933-177717718cb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:56.089397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2697e-ae85-4eab-9933-177717718cb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:56.090838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2697e-ae85-4eab-9933-177717718cb2 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:56.097957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db498f-6cfa-456d-96d6-21c93d0f9cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:56.098560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db498f-6cfa-456d-96d6-21c93d0f9cca None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:56.098839 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db498f-6cfa-456d-96d6-21c93d0f9cca None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:56.099143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db498f-6cfa-456d-96d6-21c93d0f9cca None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:56.099825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db498f-6cfa-456d-96d6-21c93d0f9cca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:56.100202 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db498f-6cfa-456d-96d6-21c93d0f9cca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:56.101635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db498f-6cfa-456d-96d6-21c93d0f9cca None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:56.103599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 326/647] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:55 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:56.104339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2697e-ae85-4eab-9933-177717718cb2 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:56.114374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 322/648] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:56.115279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db498f-6cfa-456d-96d6-21c93d0f9cca None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:56.169033 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:56.217115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:56.217943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2697e-ae85-4eab-9933-177717718cb2 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.222914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db498f-6cfa-456d-96d6-21c93d0f9cca None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.251594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65434b3c-c6b1-44de-9568-e6897fa5fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:56.253086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65434b3c-c6b1-44de-9568-e6897fa5fc12 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:56.253372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65434b3c-c6b1-44de-9568-e6897fa5fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:56.253648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65434b3c-c6b1-44de-9568-e6897fa5fc12 None admin] PATH_INFO: `/v3/users/glance` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:56.260927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136c17-ff0d-45fd-a29b-ce4a8e025cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:56.261445 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65434b3c-c6b1-44de-9568-e6897fa5fc12 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:56.261552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136c17-ff0d-45fd-a29b-ce4a8e025cbb None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:56.261802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136c17-ff0d-45fd-a29b-ce4a8e025cbb None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:56.262123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76136c17-ff0d-45fd-a29b-ce4a8e025cbb None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:56.262765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76136c17-ff0d-45fd-a29b-ce4a8e025cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:56.262935 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65434b3c-c6b1-44de-9568-e6897fa5fc12 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:56.263135 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76136c17-ff0d-45fd-a29b-ce4a8e025cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:56.264581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76136c17-ff0d-45fd-a29b-ce4a8e025cbb None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-65434b3c-c6b1-44de-9568-e6897fa5fc12 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:56.274113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:56.275280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 327/649] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:56.276247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65434b3c-c6b1-44de-9568-e6897fa5fc12 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:56.282678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 323/650] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:56.283764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136c17-ff0d-45fd-a29b-ce4a8e025cbb None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:56.389520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65434b3c-c6b1-44de-9568-e6897fa5fc12 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.398751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76136c17-ff0d-45fd-a29b-ce4a8e025cbb None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.414414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:56.427752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958f3a8-c53f-4ebb-807a-1e09c51866e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:56.427752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958f3a8-c53f-4ebb-807a-1e09c51866e4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:56.427752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958f3a8-c53f-4ebb-807a-1e09c51866e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:56.428274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958f3a8-c53f-4ebb-807a-1e09c51866e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:56.435350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239648c-a39e-4fd6-b72f-157189ce6f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:56.436069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239648c-a39e-4fd6-b72f-157189ce6f63 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:56.436345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239648c-a39e-4fd6-b72f-157189ce6f63 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:56.436618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239648c-a39e-4fd6-b72f-157189ce6f63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:56.437336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239648c-a39e-4fd6-b72f-157189ce6f63 None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=3c822c8f23b74bae84769046fdb85db4) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:56.437640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239648c-a39e-4fd6-b72f-157189ce6f63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:56.438816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958f3a8-c53f-4ebb-807a-1e09c51866e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:56.439065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239648c-a39e-4fd6-b72f-157189ce6f63 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:56.440856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958f3a8-c53f-4ebb-807a-1e09c51866e4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:56.455051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 324/651] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:56 2022] GET /identity/v3/role_assignments?user.id=3c822c8f23b74bae84769046fdb85db4&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 610 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:56.455051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239648c-a39e-4fd6-b72f-157189ce6f63 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-2958f3a8-c53f-4ebb-807a-1e09c51866e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:56.465171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:56.466282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:56.467279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:56.467279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:56.467279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:56.467279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 328/652] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:56.467279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958f3a8-c53f-4ebb-807a-1e09c51866e4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:56.487632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:56.590799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239648c-a39e-4fd6-b72f-157189ce6f63 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.594464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958f3a8-c53f-4ebb-807a-1e09c51866e4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.638997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791824a7-7756-49f5-b9ba-7a57823cbc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:56.639986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791824a7-7756-49f5-b9ba-7a57823cbc11 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:56.640263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791824a7-7756-49f5-b9ba-7a57823cbc11 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:56.640532 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791824a7-7756-49f5-b9ba-7a57823cbc11 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:56.641107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea09e28-07bb-420d-a7b9-12e6009a6b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:56.641239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791824a7-7756-49f5-b9ba-7a57823cbc11 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:56.641541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791824a7-7756-49f5-b9ba-7a57823cbc11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:56.641729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea09e28-07bb-420d-a7b9-12e6009a6b49 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:56.642001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea09e28-07bb-420d-a7b9-12e6009a6b49 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:56.642265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea09e28-07bb-420d-a7b9-12e6009a6b49 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:56.643200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791824a7-7756-49f5-b9ba-7a57823cbc11 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:56.651054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea09e28-07bb-420d-a7b9-12e6009a6b49 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:56.652514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea09e28-07bb-420d-a7b9-12e6009a6b49 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:56.662590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 325/653] 10.222.0.105 () {56 vars in 1142 bytes} [Wed Aug 24 04:41:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:56.663272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791824a7-7756-49f5-b9ba-7a57823cbc11 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-bea09e28-07bb-420d-a7b9-12e6009a6b49 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:56.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:56.666534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:56.668006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 329/654] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:56.679072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea09e28-07bb-420d-a7b9-12e6009a6b49 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:56.719104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:56.765273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:56.796192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791824a7-7756-49f5-b9ba-7a57823cbc11 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.805806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea09e28-07bb-420d-a7b9-12e6009a6b49 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.841752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f89c46-1720-4c5a-8c2a-0c4ce84006cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:56.842428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc58c0-1e19-4d20-b333-e8697c057222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:56.842878 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f89c46-1720-4c5a-8c2a-0c4ce84006cc None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:56.843248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc58c0-1e19-4d20-b333-e8697c057222 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:56.843596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc58c0-1e19-4d20-b333-e8697c057222 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:56.843767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f89c46-1720-4c5a-8c2a-0c4ce84006cc None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:56.844010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc58c0-1e19-4d20-b333-e8697c057222 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:56.844396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f89c46-1720-4c5a-8c2a-0c4ce84006cc None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:56.844908 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc58c0-1e19-4d20-b333-e8697c057222 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:56.845279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc58c0-1e19-4d20-b333-e8697c057222 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:56.846431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f89c46-1720-4c5a-8c2a-0c4ce84006cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:56.847060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f89c46-1720-4c5a-8c2a-0c4ce84006cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:56.847647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fc58c0-1e19-4d20-b333-e8697c057222 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:56.848816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f89c46-1720-4c5a-8c2a-0c4ce84006cc None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:56.864864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 326/655] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:56.866042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f89c46-1720-4c5a-8c2a-0c4ce84006cc None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:56.869303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 330/656] 10.222.0.105 () {56 vars in 1142 bytes} [Wed Aug 24 04:41:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:56.873631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc58c0-1e19-4d20-b333-e8697c057222 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:56.979960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fc58c0-1e19-4d20-b333-e8697c057222 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:56.983336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:56.994174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f89c46-1720-4c5a-8c2a-0c4ce84006cc None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:57.018919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a329c-2cf3-4325-bc37-a6f2af15408b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:57.020159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a329c-2cf3-4325-bc37-a6f2af15408b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:57.020505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a329c-2cf3-4325-bc37-a6f2af15408b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:57.020991 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7a329c-2cf3-4325-bc37-a6f2af15408b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:57.030668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a329c-2cf3-4325-bc37-a6f2af15408b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:57.031495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad4917-b0fc-484f-afe2-4c5081926f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:57.032134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7a329c-2cf3-4325-bc37-a6f2af15408b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:57.032642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4917-b0fc-484f-afe2-4c5081926f57 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:57.033153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4917-b0fc-484f-afe2-4c5081926f57 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:57.033631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad4917-b0fc-484f-afe2-4c5081926f57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:57.044021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-1b7a329c-2cf3-4325-bc37-a6f2af15408b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:57.047900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:57.049364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:57.050335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:57.050335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:57.050335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:57.050335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:57.050335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:57.050335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:57.050335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:57.050335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:57.050335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 04:41:57.050335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:57.054738 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 331/657] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:57.055658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a329c-2cf3-4325-bc37-a6f2af15408b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:57.129846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad4917-b0fc-484f-afe2-4c5081926f57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:57.131424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad4917-b0fc-484f-afe2-4c5081926f57 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:57.180859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7a329c-2cf3-4325-bc37-a6f2af15408b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:57.222349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df24b66-5143-4515-913e-1a792f79a0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:57.223139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df24b66-5143-4515-913e-1a792f79a0b8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:57.223416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df24b66-5143-4515-913e-1a792f79a0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:57.223754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df24b66-5143-4515-913e-1a792f79a0b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:57.243453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 327/658] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:56 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:57.244481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad4917-b0fc-484f-afe2-4c5081926f57 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:57.250726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:57.299617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df24b66-5143-4515-913e-1a792f79a0b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:57.299853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:57.301224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df24b66-5143-4515-913e-1a792f79a0b8 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:57.348812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad4917-b0fc-484f-afe2-4c5081926f57 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:57.384905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca15498-ecdd-490b-b7b1-516d7aacb56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:57.385511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca15498-ecdd-490b-b7b1-516d7aacb56d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:57.385787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca15498-ecdd-490b-b7b1-516d7aacb56d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:57.386057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca15498-ecdd-490b-b7b1-516d7aacb56d None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:57.386778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca15498-ecdd-490b-b7b1-516d7aacb56d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:57.387117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca15498-ecdd-490b-b7b1-516d7aacb56d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:57.388545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca15498-ecdd-490b-b7b1-516d7aacb56d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:57.407321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 328/659] 10.222.0.105 () {56 vars in 1144 bytes} [Wed Aug 24 04:41:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:57.407956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca15498-ecdd-490b-b7b1-516d7aacb56d None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:57.417893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 332/660] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:57 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:57.417893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907be6d5-a79b-47e7-8387-30eade5b3b26 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:57.417893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907be6d5-a79b-47e7-8387-30eade5b3b26 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:57.417893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907be6d5-a79b-47e7-8387-30eade5b3b26 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:57.417893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 333/661] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:57 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:57.417893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907be6d5-a79b-47e7-8387-30eade5b3b26 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:57.476977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:57.511987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:57.520500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca15498-ecdd-490b-b7b1-516d7aacb56d None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:57.525744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907be6d5-a79b-47e7-8387-30eade5b3b26 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:57.559206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1890f2-fd9b-4608-9b48-9a881b4d0e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:57.559913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1890f2-fd9b-4608-9b48-9a881b4d0e38 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:57.560162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1890f2-fd9b-4608-9b48-9a881b4d0e38 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:57.560438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1890f2-fd9b-4608-9b48-9a881b4d0e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:57.561539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5dc89-b702-4a01-9ba7-47b733d31563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:57.562184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5dc89-b702-4a01-9ba7-47b733d31563 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:57.562577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5dc89-b702-4a01-9ba7-47b733d31563 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:57.562847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5dc89-b702-4a01-9ba7-47b733d31563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:57.571418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1890f2-fd9b-4608-9b48-9a881b4d0e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:57.573222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1890f2-fd9b-4608-9b48-9a881b4d0e38 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-4c1890f2-fd9b-4608-9b48-9a881b4d0e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:57.589665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:57.590631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:57.591590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:57.591590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:57.591590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:57.591590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 329/662] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:57.591840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1890f2-fd9b-4608-9b48-9a881b4d0e38 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:57.639202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5dc89-b702-4a01-9ba7-47b733d31563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:57.640648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5dc89-b702-4a01-9ba7-47b733d31563 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:57.696591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:57.701658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1890f2-fd9b-4608-9b48-9a881b4d0e38 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:57.737555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1220ef4-32ad-4f55-af12-a8c41702f0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:57.738184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1220ef4-32ad-4f55-af12-a8c41702f0fe None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:57.738469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1220ef4-32ad-4f55-af12-a8c41702f0fe None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:57.738739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1220ef4-32ad-4f55-af12-a8c41702f0fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:57.743798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 334/663] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:57 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:41:57.744918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24c57d-eae7-441c-90b4-d2b854864ded None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:57.745204 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24c57d-eae7-441c-90b4-d2b854864ded None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:57.745477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24c57d-eae7-441c-90b4-d2b854864ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:57.755662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1220ef4-32ad-4f55-af12-a8c41702f0fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:57.757187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1220ef4-32ad-4f55-af12-a8c41702f0fe None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:57.764593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-0e24c57d-eae7-441c-90b4-d2b854864ded None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:57.765761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0e24c57d-eae7-441c-90b4-d2b854864ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:57.767882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-b1220ef4-32ad-4f55-af12-a8c41702f0fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:57.774587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:57.781325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:57.782264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:57.782264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:57.782264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:57.782264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 330/664] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:57.782264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1220ef4-32ad-4f55-af12-a8c41702f0fe None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:57.847062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 335/665] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:57.847062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24c57d-eae7-441c-90b4-d2b854864ded None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:57.885934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1220ef4-32ad-4f55-af12-a8c41702f0fe None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:57.925650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b0a36-e0e2-445a-9aad-a91c64d57713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:57.926523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b0a36-e0e2-445a-9aad-a91c64d57713 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:57.926951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b0a36-e0e2-445a-9aad-a91c64d57713 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:57.927407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b0a36-e0e2-445a-9aad-a91c64d57713 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:57.928199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b0a36-e0e2-445a-9aad-a91c64d57713 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:57.928649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b0a36-e0e2-445a-9aad-a91c64d57713 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:57.930159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b0a36-e0e2-445a-9aad-a91c64d57713 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:57.947961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 331/666] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:57.949075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b0a36-e0e2-445a-9aad-a91c64d57713 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:57.956712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:57.978444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24c57d-eae7-441c-90b4-d2b854864ded None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.016971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:58.025952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d0cde-0f19-4079-9c64-0a5a18812bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.031419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0cde-0f19-4079-9c64-0a5a18812bde None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.031419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0cde-0f19-4079-9c64-0a5a18812bde None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.031419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d0cde-0f19-4079-9c64-0a5a18812bde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.040931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d0cde-0f19-4079-9c64-0a5a18812bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.043078 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d0cde-0f19-4079-9c64-0a5a18812bde None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.063795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b0a36-e0e2-445a-9aad-a91c64d57713 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-533d0cde-0f19-4079-9c64-0a5a18812bde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:58.063963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:58.065430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:58.066333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:58.066333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:58.066333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:58.066333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 336/667] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:58.067039 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d0cde-0f19-4079-9c64-0a5a18812bde None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.113945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685d75d-a53a-416e-8be7-3533f8cea0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.114557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685d75d-a53a-416e-8be7-3533f8cea0e5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.114837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685d75d-a53a-416e-8be7-3533f8cea0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.115136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685d75d-a53a-416e-8be7-3533f8cea0e5 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.115774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685d75d-a53a-416e-8be7-3533f8cea0e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:58.116074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685d75d-a53a-416e-8be7-3533f8cea0e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.118020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685d75d-a53a-416e-8be7-3533f8cea0e5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.134431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 332/668] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:58.135632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685d75d-a53a-416e-8be7-3533f8cea0e5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.185912 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d0cde-0f19-4079-9c64-0a5a18812bde None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.224976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79188a42-74b0-48e8-bd3b-650b023f7414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.225607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79188a42-74b0-48e8-bd3b-650b023f7414 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.225891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79188a42-74b0-48e8-bd3b-650b023f7414 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.226164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79188a42-74b0-48e8-bd3b-650b023f7414 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.229071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:58.233835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79188a42-74b0-48e8-bd3b-650b023f7414 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.235297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79188a42-74b0-48e8-bd3b-650b023f7414 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-79188a42-74b0-48e8-bd3b-650b023f7414 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:58.248030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:58.248960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:58.249843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:58.249843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:58.249843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:58.249843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:58.249843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:58.249843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:58.249843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:58.249843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:58.249843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 337/669] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:58.250342 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79188a42-74b0-48e8-bd3b-650b023f7414 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.256690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685d75d-a53a-416e-8be7-3533f8cea0e5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.277831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:58.293529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c968f8-2e31-44f2-8a9f-93a93f3d77ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.294147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c968f8-2e31-44f2-8a9f-93a93f3d77ea None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.294428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c968f8-2e31-44f2-8a9f-93a93f3d77ea None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.294702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c968f8-2e31-44f2-8a9f-93a93f3d77ea None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.295372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c968f8-2e31-44f2-8a9f-93a93f3d77ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:58.295674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c968f8-2e31-44f2-8a9f-93a93f3d77ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.297038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c968f8-2e31-44f2-8a9f-93a93f3d77ea None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.313752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 333/670] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:58.313752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c968f8-2e31-44f2-8a9f-93a93f3d77ea None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.361056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79188a42-74b0-48e8-bd3b-650b023f7414 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.397258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fca84f-a317-4acd-80ee-8ba4c96fb0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.398004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fca84f-a317-4acd-80ee-8ba4c96fb0ba None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.398281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fca84f-a317-4acd-80ee-8ba4c96fb0ba None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.398550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fca84f-a317-4acd-80ee-8ba4c96fb0ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.409689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fca84f-a317-4acd-80ee-8ba4c96fb0ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-47fca84f-a317-4acd-80ee-8ba4c96fb0ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:58.414817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:58.415765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:58.416616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 338/671] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:58.417353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fca84f-a317-4acd-80ee-8ba4c96fb0ba None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.424960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c968f8-2e31-44f2-8a9f-93a93f3d77ea None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.461477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56069589-8d84-46ad-af44-d42f5a6809af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.462063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56069589-8d84-46ad-af44-d42f5a6809af None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.462342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56069589-8d84-46ad-af44-d42f5a6809af None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.462612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56069589-8d84-46ad-af44-d42f5a6809af None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.469461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56069589-8d84-46ad-af44-d42f5a6809af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.470927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56069589-8d84-46ad-af44-d42f5a6809af None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.476161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-56069589-8d84-46ad-af44-d42f5a6809af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:58.482939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:58.490147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:58.491303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:58.491303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:58.491303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:58.491303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:58.491303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:58.491303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:58.491303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:58.491303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:58.491303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 334/672] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:58.491303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56069589-8d84-46ad-af44-d42f5a6809af None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.528340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fca84f-a317-4acd-80ee-8ba4c96fb0ba None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.531479 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:58.566643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718ed69-1dd0-415c-af61-83368fb63ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.567517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718ed69-1dd0-415c-af61-83368fb63ca2 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.567946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718ed69-1dd0-415c-af61-83368fb63ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.568373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718ed69-1dd0-415c-af61-83368fb63ca2 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.569353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718ed69-1dd0-415c-af61-83368fb63ca2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:58.569814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718ed69-1dd0-415c-af61-83368fb63ca2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.571783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718ed69-1dd0-415c-af61-83368fb63ca2 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.584736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 339/673] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:58.585844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718ed69-1dd0-415c-af61-83368fb63ca2 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.599055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56069589-8d84-46ad-af44-d42f5a6809af None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.634459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4f20f-75af-43d6-b103-0033c863c99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.635121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4f20f-75af-43d6-b103-0033c863c99e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.635405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4f20f-75af-43d6-b103-0033c863c99e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.635696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4f20f-75af-43d6-b103-0033c863c99e None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.642390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4f20f-75af-43d6-b103-0033c863c99e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.643863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4f20f-75af-43d6-b103-0033c863c99e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-48b4f20f-75af-43d6-b103-0033c863c99e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:41:58.655861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:58.656746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:58.658182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:41:58.658182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:41:58.658182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:41:58.658182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:41:58.658182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:41:58.658182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:41:58.658182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:41:58.658182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:58.658182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 335/674] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:58.658876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4f20f-75af-43d6-b103-0033c863c99e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.695185 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718ed69-1dd0-415c-af61-83368fb63ca2 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.711269 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:58.729097 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fd29e-2bc5-4cbc-b476-2d13d388fba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.729722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fd29e-2bc5-4cbc-b476-2d13d388fba8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.730000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fd29e-2bc5-4cbc-b476-2d13d388fba8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.730297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26fd29e-2bc5-4cbc-b476-2d13d388fba8 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.730948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fd29e-2bc5-4cbc-b476-2d13d388fba8 None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:58.731276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fd29e-2bc5-4cbc-b476-2d13d388fba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.732632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26fd29e-2bc5-4cbc-b476-2d13d388fba8 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.742388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 340/675] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:58.743222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fd29e-2bc5-4cbc-b476-2d13d388fba8 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.755110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:58.757978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4f20f-75af-43d6-b103-0033c863c99e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.791561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e760b36-e6d7-4f65-b2eb-3687e3da5c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.792085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e760b36-e6d7-4f65-b2eb-3687e3da5c7f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.792362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e760b36-e6d7-4f65-b2eb-3687e3da5c7f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.792630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e760b36-e6d7-4f65-b2eb-3687e3da5c7f None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.793304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e760b36-e6d7-4f65-b2eb-3687e3da5c7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:58.793602 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e760b36-e6d7-4f65-b2eb-3687e3da5c7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.795036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e760b36-e6d7-4f65-b2eb-3687e3da5c7f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.806742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 336/676] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:58.808090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbb59b-e618-4939-afa4-15d046f15893 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.808450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbb59b-e618-4939-afa4-15d046f15893 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.808729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dbb59b-e618-4939-afa4-15d046f15893 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.811093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 337/677] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:41:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:41:58.811093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbb59b-e618-4939-afa4-15d046f15893 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.844648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26fd29e-2bc5-4cbc-b476-2d13d388fba8 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.877788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057d69-af3c-49f8-b55f-d040315cc48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.878385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057d69-af3c-49f8-b55f-d040315cc48c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.878662 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057d69-af3c-49f8-b55f-d040315cc48c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.878929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60057d69-af3c-49f8-b55f-d040315cc48c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.879682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60057d69-af3c-49f8-b55f-d040315cc48c None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=f1d62a4c07c4402ea5803f6e68fb1d0b) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:58.879989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60057d69-af3c-49f8-b55f-d040315cc48c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.881401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60057d69-af3c-49f8-b55f-d040315cc48c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.886996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 341/678] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/role_assignments?user.id=f1d62a4c07c4402ea5803f6e68fb1d0b&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 610 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:58.887793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057d69-af3c-49f8-b55f-d040315cc48c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:58.920679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dbb59b-e618-4939-afa4-15d046f15893 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:58.947811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:58.957915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1bca3-ed80-4ab2-a3b9-fd8b499970e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:58.967654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1bca3-ed80-4ab2-a3b9-fd8b499970e7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.967654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1bca3-ed80-4ab2-a3b9-fd8b499970e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.967654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1bca3-ed80-4ab2-a3b9-fd8b499970e7 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.967654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1bca3-ed80-4ab2-a3b9-fd8b499970e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:58.967654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1bca3-ed80-4ab2-a3b9-fd8b499970e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:58.967654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1bca3-ed80-4ab2-a3b9-fd8b499970e7 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:58.976183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 338/679] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:58.977536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773aa1c-6632-49e0-be6b-14b44e3c43d7 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:58.977823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773aa1c-6632-49e0-be6b-14b44e3c43d7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:58.978085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773aa1c-6632-49e0-be6b-14b44e3c43d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:58.997712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-4773aa1c-6632-49e0-be6b-14b44e3c43d7 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:41:58.998941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4773aa1c-6632-49e0-be6b-14b44e3c43d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:41:59.015915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:59.022366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60057d69-af3c-49f8-b55f-d040315cc48c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:59.067257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3ba70-59e7-4ec5-81b2-3f9fcc4860ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:59.068051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3ba70-59e7-4ec5-81b2-3f9fcc4860ec None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:59.068328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3ba70-59e7-4ec5-81b2-3f9fcc4860ec None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:59.068595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3ba70-59e7-4ec5-81b2-3f9fcc4860ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:59.082536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3ba70-59e7-4ec5-81b2-3f9fcc4860ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:59.082993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 339/680] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:41:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:41:59.083886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773aa1c-6632-49e0-be6b-14b44e3c43d7 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:59.084293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3ba70-59e7-4ec5-81b2-3f9fcc4860ec None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-28e3ba70-59e7-4ec5-81b2-3f9fcc4860ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:41:59.099788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:59.100757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:41:59.101697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:41:59.101697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:41:59.101697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:59.105614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 342/681] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:41:59.106440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3ba70-59e7-4ec5-81b2-3f9fcc4860ec None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:59.218356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773aa1c-6632-49e0-be6b-14b44e3c43d7 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:59.242565 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:59.254437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3ba70-59e7-4ec5-81b2-3f9fcc4860ec None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:59.259309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd67a24-67f7-4714-99c6-11c101bc765d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:59.259309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd67a24-67f7-4714-99c6-11c101bc765d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:59.259309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd67a24-67f7-4714-99c6-11c101bc765d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:59.259309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd67a24-67f7-4714-99c6-11c101bc765d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:59.259685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd67a24-67f7-4714-99c6-11c101bc765d None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=29af87e1e05b4880b9ec96865994b635) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:59.259685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd67a24-67f7-4714-99c6-11c101bc765d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:59.260334 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd67a24-67f7-4714-99c6-11c101bc765d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:59.267275 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 340/682] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:59 2022] GET /identity/v3/role_assignments?user.id=29af87e1e05b4880b9ec96865994b635&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 610 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:59.268211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd67a24-67f7-4714-99c6-11c101bc765d None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:59.285884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:59.307129 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b188297-0077-4f6c-ad0a-1acb4148aeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:59.312437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b188297-0077-4f6c-ad0a-1acb4148aeeb None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:59.313009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b188297-0077-4f6c-ad0a-1acb4148aeeb None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:59.313426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b188297-0077-4f6c-ad0a-1acb4148aeeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:59.315422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b188297-0077-4f6c-ad0a-1acb4148aeeb None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=44d2b4015ac74b8b8d0a674419f34cf7) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:59.315928 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b188297-0077-4f6c-ad0a-1acb4148aeeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:59.318246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b188297-0077-4f6c-ad0a-1acb4148aeeb None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:59.327380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 343/683] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:41:59 2022] GET /identity/v3/role_assignments?user.id=44d2b4015ac74b8b8d0a674419f34cf7&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 610 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:59.389922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd67a24-67f7-4714-99c6-11c101bc765d None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:59.397668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b188297-0077-4f6c-ad0a-1acb4148aeeb None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:59.432022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f59b0-6fd5-4476-8ebb-d5841260dd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:59.432888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f59b0-6fd5-4476-8ebb-d5841260dd3b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:59.433203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f59b0-6fd5-4476-8ebb-d5841260dd3b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:59.433484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f59b0-6fd5-4476-8ebb-d5841260dd3b None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:59.434157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f59b0-6fd5-4476-8ebb-d5841260dd3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:59.434465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f59b0-6fd5-4476-8ebb-d5841260dd3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:59.435930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f59b0-6fd5-4476-8ebb-d5841260dd3b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:59.451212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 341/684] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:41:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:59.458454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f59b0-6fd5-4476-8ebb-d5841260dd3b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:59.567213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:59.583067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b188297-0077-4f6c-ad0a-1acb4148aeeb None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:59.594591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f59b0-6fd5-4476-8ebb-d5841260dd3b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:59.624143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbcbe6-91e2-44aa-8854-433d0112ff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:59.624841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbcbe6-91e2-44aa-8854-433d0112ff77 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:59.625124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbcbe6-91e2-44aa-8854-433d0112ff77 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:59.625393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbcbe6-91e2-44aa-8854-433d0112ff77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:59.641780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc0f8-cc1b-4dc5-9651-b41ca8249c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:59.642941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc0f8-cc1b-4dc5-9651-b41ca8249c3d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:59.643240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc0f8-cc1b-4dc5-9651-b41ca8249c3d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:59.643509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1bc0f8-cc1b-4dc5-9651-b41ca8249c3d None admin] PATH_INFO: `/v3/users/swift` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:59.644705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbcbe6-91e2-44aa-8854-433d0112ff77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:41:59.651725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bc0f8-cc1b-4dc5-9651-b41ca8249c3d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-14bbcbe6-91e2-44aa-8854-433d0112ff77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:41:59.652163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:59.653118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:41:59.653986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:41:59.654642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1bc0f8-cc1b-4dc5-9651-b41ca8249c3d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:59.656564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 344/685] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:41:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:59.664701 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbcbe6-91e2-44aa-8854-433d0112ff77 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-bb1bc0f8-cc1b-4dc5-9651-b41ca8249c3d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:41:59.666375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:41:59.667391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:41:59.668398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 342/686] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:41:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:41:59.674426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc0f8-cc1b-4dc5-9651-b41ca8249c3d None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:59.691268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:59.779978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbcbe6-91e2-44aa-8854-433d0112ff77 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:59.807568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1bc0f8-cc1b-4dc5-9651-b41ca8249c3d None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:41:59.839222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac877659-eba3-4572-91bb-b8e2d0c88c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:59.839910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac877659-eba3-4572-91bb-b8e2d0c88c1e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:59.840290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac877659-eba3-4572-91bb-b8e2d0c88c1e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:59.840684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac877659-eba3-4572-91bb-b8e2d0c88c1e None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:59.841556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac877659-eba3-4572-91bb-b8e2d0c88c1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:59.841988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac877659-eba3-4572-91bb-b8e2d0c88c1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:59.843570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac877659-eba3-4572-91bb-b8e2d0c88c1e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:59.858753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf422134-2615-47fb-816f-10d3b328e4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:41:59.859604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf422134-2615-47fb-816f-10d3b328e4ab None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:41:59.860028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf422134-2615-47fb-816f-10d3b328e4ab None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:41:59.860968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf422134-2615-47fb-816f-10d3b328e4ab None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:41:59.863159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 345/687] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:41:59 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:59.864076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf422134-2615-47fb-816f-10d3b328e4ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:41:59.864534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf422134-2615-47fb-816f-10d3b328e4ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:41:59.866110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf422134-2615-47fb-816f-10d3b328e4ab None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:41:59.868686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:41:59.869905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac877659-eba3-4572-91bb-b8e2d0c88c1e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:41:59.902967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 343/688] 10.222.0.105 () {56 vars in 1140 bytes} [Wed Aug 24 04:41:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:41:59.909443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf422134-2615-47fb-816f-10d3b328e4ab None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:00.050776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf422134-2615-47fb-816f-10d3b328e4ab None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:00.053002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:00.089450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac877659-eba3-4572-91bb-b8e2d0c88c1e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:00.117223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784065d7-1d03-4faa-b1fe-de2bad268689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:00.118720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784065d7-1d03-4faa-b1fe-de2bad268689 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:00.120500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784065d7-1d03-4faa-b1fe-de2bad268689 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:00.120981 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784065d7-1d03-4faa-b1fe-de2bad268689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:00.153922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72821ec0-6acc-452c-8aa6-49c56d5c18a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:00.154910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72821ec0-6acc-452c-8aa6-49c56d5c18a9 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:00.155599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72821ec0-6acc-452c-8aa6-49c56d5c18a9 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:00.158682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72821ec0-6acc-452c-8aa6-49c56d5c18a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:00.180481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72821ec0-6acc-452c-8aa6-49c56d5c18a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:00.182276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72821ec0-6acc-452c-8aa6-49c56d5c18a9 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-72821ec0-6acc-452c-8aa6-49c56d5c18a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:00.210416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:00.211419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:00.212344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:00.212344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:00.212344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:00.222688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 346/689] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:00.224064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72821ec0-6acc-452c-8aa6-49c56d5c18a9 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:00.239423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:00.249038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784065d7-1d03-4faa-b1fe-de2bad268689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:00.251473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784065d7-1d03-4faa-b1fe-de2bad268689 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:00.406261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72821ec0-6acc-452c-8aa6-49c56d5c18a9 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:00.414511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 344/690] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:41:59 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:42:00.420195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784065d7-1d03-4faa-b1fe-de2bad268689 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:00.435620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:00.459703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ce4b-d4c2-4ab9-8080-a48ec46ee38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:00.460365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ce4b-d4c2-4ab9-8080-a48ec46ee38e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:00.460661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ce4b-d4c2-4ab9-8080-a48ec46ee38e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:00.460930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb89ce4b-d4c2-4ab9-8080-a48ec46ee38e None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:00.461568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89ce4b-d4c2-4ab9-8080-a48ec46ee38e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:00.461872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89ce4b-d4c2-4ab9-8080-a48ec46ee38e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:00.463322 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb89ce4b-d4c2-4ab9-8080-a48ec46ee38e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:00.492781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 347/691] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:00.497624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ce4b-d4c2-4ab9-8080-a48ec46ee38e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:00.559403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:00.575769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784065d7-1d03-4faa-b1fe-de2bad268689 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:00.649200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929f98b-8a40-4266-9470-05bb784032ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:00.649830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929f98b-8a40-4266-9470-05bb784032ed None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:00.650101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929f98b-8a40-4266-9470-05bb784032ed None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:00.650363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929f98b-8a40-4266-9470-05bb784032ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:00.669128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929f98b-8a40-4266-9470-05bb784032ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:00.670497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929f98b-8a40-4266-9470-05bb784032ed None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:00.688323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb89ce4b-d4c2-4ab9-8080-a48ec46ee38e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-a929f98b-8a40-4266-9470-05bb784032ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:00.702713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:00.703779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:00.705310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:00.705310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:00.705310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:00.706703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 345/692] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:00.711310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929f98b-8a40-4266-9470-05bb784032ed None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:00.762842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78601526-ef88-4874-a417-28d5b8a372a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:00.767721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78601526-ef88-4874-a417-28d5b8a372a8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:00.768048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78601526-ef88-4874-a417-28d5b8a372a8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:00.768271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78601526-ef88-4874-a417-28d5b8a372a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:00.777056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78601526-ef88-4874-a417-28d5b8a372a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:00.778499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78601526-ef88-4874-a417-28d5b8a372a8 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-78601526-ef88-4874-a417-28d5b8a372a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:00.805340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:00.806345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:00.807330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:00.807330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:00.807330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:00.807330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:00.807330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:00.807330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:00.807330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:00.807330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:00.807330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:00.807330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:00.810548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 348/693] 10.222.0.105 () {56 vars in 1098 bytes} [Wed Aug 24 04:42:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:00.820092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78601526-ef88-4874-a417-28d5b8a372a8 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:00.825432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929f98b-8a40-4266-9470-05bb784032ed None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:00.863731 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa06df-9361-48e9-a3cf-a53fd9a8ec2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:00.864340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa06df-9361-48e9-a3cf-a53fd9a8ec2e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:00.864613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa06df-9361-48e9-a3cf-a53fd9a8ec2e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:00.864877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa06df-9361-48e9-a3cf-a53fd9a8ec2e None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:00.865510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa06df-9361-48e9-a3cf-a53fd9a8ec2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:00.865804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa06df-9361-48e9-a3cf-a53fd9a8ec2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:00.867172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aa06df-9361-48e9-a3cf-a53fd9a8ec2e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:00.882351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 346/694] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:00.892371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa06df-9361-48e9-a3cf-a53fd9a8ec2e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:00.903336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:00.910759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:00.961264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78601526-ef88-4874-a417-28d5b8a372a8 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:00.997147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad776a92-ae59-4a94-9fa3-5e9bb8b0af66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:00.997754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad776a92-ae59-4a94-9fa3-5e9bb8b0af66 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:00.998031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad776a92-ae59-4a94-9fa3-5e9bb8b0af66 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:00.998299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad776a92-ae59-4a94-9fa3-5e9bb8b0af66 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:00.999029 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad776a92-ae59-4a94-9fa3-5e9bb8b0af66 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:00.999396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad776a92-ae59-4a94-9fa3-5e9bb8b0af66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:01.001346 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad776a92-ae59-4a94-9fa3-5e9bb8b0af66 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:01.022094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 349/695] 10.222.0.105 () {56 vars in 1138 bytes} [Wed Aug 24 04:42:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:01.028062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad776a92-ae59-4a94-9fa3-5e9bb8b0af66 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:01.046134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa06df-9361-48e9-a3cf-a53fd9a8ec2e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:01.087391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d129b58-6a53-4ae2-9259-58a9edbff390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:01.088153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d129b58-6a53-4ae2-9259-58a9edbff390 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.088490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d129b58-6a53-4ae2-9259-58a9edbff390 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.088702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d129b58-6a53-4ae2-9259-58a9edbff390 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.096839 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d129b58-6a53-4ae2-9259-58a9edbff390 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:01.098941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d129b58-6a53-4ae2-9259-58a9edbff390 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-7d129b58-6a53-4ae2-9259-58a9edbff390 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:01.112249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:01.113173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:01.114678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:01.114678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:01.114678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:01.114678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:01.114678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:01.114678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:01.114678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:01.114678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:01.114678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 347/696] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:01.119217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d129b58-6a53-4ae2-9259-58a9edbff390 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:01.221696 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad776a92-ae59-4a94-9fa3-5e9bb8b0af66 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:01.229957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:01.237729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:01.269095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849eeba5-5061-4dc3-bd06-3fb758ec1a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:01.270451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849eeba5-5061-4dc3-bd06-3fb758ec1a40 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.270980 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849eeba5-5061-4dc3-bd06-3fb758ec1a40 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.271450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849eeba5-5061-4dc3-bd06-3fb758ec1a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.280440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d129b58-6a53-4ae2-9259-58a9edbff390 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:01.328572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aab7b-9002-4ff8-ad37-0924ae09fea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:01.329689 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aab7b-9002-4ff8-ad37-0924ae09fea1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.330069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aab7b-9002-4ff8-ad37-0924ae09fea1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.330435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aab7b-9002-4ff8-ad37-0924ae09fea1 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.331498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aab7b-9002-4ff8-ad37-0924ae09fea1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:01.331919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aab7b-9002-4ff8-ad37-0924ae09fea1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:01.334219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aab7b-9002-4ff8-ad37-0924ae09fea1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:01.356794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 348/697] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:01.356794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e905b-1b5e-492f-a180-f605761e7a32 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.356794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e905b-1b5e-492f-a180-f605761e7a32 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.356794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e905b-1b5e-492f-a180-f605761e7a32 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.356794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 349/698] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:01.358176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e905b-1b5e-492f-a180-f605761e7a32 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:01.367677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849eeba5-5061-4dc3-bd06-3fb758ec1a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:01.369131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849eeba5-5061-4dc3-bd06-3fb758ec1a40 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:01.502951 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 350/699] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:42:01.504351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e905b-1b5e-492f-a180-f605761e7a32 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:01.505452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af3c8e-b9b0-4149-8264-6aa683d3841e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.505920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af3c8e-b9b0-4149-8264-6aa683d3841e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.506423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af3c8e-b9b0-4149-8264-6aa683d3841e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.524290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b3af3c8e-b9b0-4149-8264-6aa683d3841e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:01.525795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b3af3c8e-b9b0-4149-8264-6aa683d3841e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:01.529760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:01.533389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:01.551696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094c479-315b-490e-8ef6-bfb1330c4825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:01.552374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c479-315b-490e-8ef6-bfb1330c4825 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.552736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c479-315b-490e-8ef6-bfb1330c4825 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.553296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094c479-315b-490e-8ef6-bfb1330c4825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.554030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094c479-315b-490e-8ef6-bfb1330c4825 None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=e6a894e4e6f24febbd8397fb8c4c7fb4) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:01.554385 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094c479-315b-490e-8ef6-bfb1330c4825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:01.556338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094c479-315b-490e-8ef6-bfb1330c4825 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:01.571054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 350/700] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:01 2022] GET /identity/v3/role_assignments?user.id=e6a894e4e6f24febbd8397fb8c4c7fb4&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 610 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:01.571054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094c479-315b-490e-8ef6-bfb1330c4825 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:01.636173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 351/701] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:42:01.691831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094c479-315b-490e-8ef6-bfb1330c4825 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:01.731069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139cafc-0338-4e75-ac67-a9c4357d7b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:01.731968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139cafc-0338-4e75-ac67-a9c4357d7b77 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.732182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139cafc-0338-4e75-ac67-a9c4357d7b77 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.732407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139cafc-0338-4e75-ac67-a9c4357d7b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.732844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080897-78cb-4365-b048-d96439d7b2e0 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.733191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080897-78cb-4365-b048-d96439d7b2e0 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.733406 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9080897-78cb-4365-b048-d96439d7b2e0 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.735546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 352/702] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:01 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:01.736548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d3bf3-ca4e-4699-8155-0e0275d3c9b9 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.736826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d3bf3-ca4e-4699-8155-0e0275d3c9b9 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.739760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d3bf3-ca4e-4699-8155-0e0275d3c9b9 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.740397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 353/703] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:01 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:01.741892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe04fd-dfd5-4320-86cc-96ffd6926b82 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.742278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe04fd-dfd5-4320-86cc-96ffd6926b82 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.742666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe04fd-dfd5-4320-86cc-96ffd6926b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.746512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139cafc-0338-4e75-ac67-a9c4357d7b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:01.748303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c139cafc-0338-4e75-ac67-a9c4357d7b77 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-c139cafc-0338-4e75-ac67-a9c4357d7b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:01.766384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:01.767454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:01.768444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:01.768444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:01.768444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:01.768444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 351/704] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:01.769134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15846a-27b0-46d5-aa00-b108a7dd590e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:01.769379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c6fe04fd-dfd5-4320-86cc-96ffd6926b82 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:01.769482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15846a-27b0-46d5-aa00-b108a7dd590e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:01.769673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f15846a-27b0-46d5-aa00-b108a7dd590e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:01.771261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c6fe04fd-dfd5-4320-86cc-96ffd6926b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:01.794511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7f15846a-27b0-46d5-aa00-b108a7dd590e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:01.795805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7f15846a-27b0-46d5-aa00-b108a7dd590e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:01.841391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:01.869275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 354/705] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:42:01.870235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe04fd-dfd5-4320-86cc-96ffd6926b82 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:01.881901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 352/706] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:42:01.941593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:01.979752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe04fd-dfd5-4320-86cc-96ffd6926b82 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:01.988235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15846a-27b0-46d5-aa00-b108a7dd590e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.021621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a79c8e-681d-4aea-8e8b-dfd1d786cc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.022335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a79c8e-681d-4aea-8e8b-dfd1d786cc55 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.022615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a79c8e-681d-4aea-8e8b-dfd1d786cc55 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.023271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a79c8e-681d-4aea-8e8b-dfd1d786cc55 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.024177 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a79c8e-681d-4aea-8e8b-dfd1d786cc55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:02.024482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a79c8e-681d-4aea-8e8b-dfd1d786cc55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.026004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a79c8e-681d-4aea-8e8b-dfd1d786cc55 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.041460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 355/707] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:02.049705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a79c8e-681d-4aea-8e8b-dfd1d786cc55 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.191056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f15846a-27b0-46d5-aa00-b108a7dd590e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:02.191683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a79c8e-681d-4aea-8e8b-dfd1d786cc55 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:02.224299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:02.230708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dc32d-b99b-472d-8392-3db04be9f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.231204 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2082c-3b8b-4dcf-a502-9f57ff4a063f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.234678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dc32d-b99b-472d-8392-3db04be9f61e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.234678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dc32d-b99b-472d-8392-3db04be9f61e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.234858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2082c-3b8b-4dcf-a502-9f57ff4a063f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.234939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41dc32d-b99b-472d-8392-3db04be9f61e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.235012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2082c-3b8b-4dcf-a502-9f57ff4a063f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.235012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2082c-3b8b-4dcf-a502-9f57ff4a063f None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.239480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2082c-3b8b-4dcf-a502-9f57ff4a063f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.240934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2082c-3b8b-4dcf-a502-9f57ff4a063f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.243477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41dc32d-b99b-472d-8392-3db04be9f61e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.244971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41dc32d-b99b-472d-8392-3db04be9f61e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-25e2082c-3b8b-4dcf-a502-9f57ff4a063f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:02.252545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:02.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:02.259629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:02.259629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:02.259629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:02.259629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:02.259629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:02.259629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:02.259629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:02.259629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:02.259629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 356/708] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:02.259629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2082c-3b8b-4dcf-a502-9f57ff4a063f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-b41dc32d-b99b-472d-8392-3db04be9f61e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:02.269463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:02.270595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:02.271722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:02.271722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:02.271722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:02.271722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 353/709] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:02.271722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dc32d-b99b-472d-8392-3db04be9f61e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.298677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:02.377357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2082c-3b8b-4dcf-a502-9f57ff4a063f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:02.380823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41dc32d-b99b-472d-8392-3db04be9f61e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:02.413918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae9cda-dad3-48ff-9575-4d42be35ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.414646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae9cda-dad3-48ff-9575-4d42be35ad88 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.414994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae9cda-dad3-48ff-9575-4d42be35ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.415289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ae9cda-dad3-48ff-9575-4d42be35ad88 None admin] PATH_INFO: `/v3/services/network` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.415863 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae9cda-dad3-48ff-9575-4d42be35ad88 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.417298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ae9cda-dad3-48ff-9575-4d42be35ad88 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.418146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0aabeb-be1c-494c-924f-6004ac8df136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.418779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aabeb-be1c-494c-924f-6004ac8df136 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.419083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aabeb-be1c-494c-924f-6004ac8df136 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.419361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0aabeb-be1c-494c-924f-6004ac8df136 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.419940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0aabeb-be1c-494c-924f-6004ac8df136 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.421282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0aabeb-be1c-494c-924f-6004ac8df136 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-d6ae9cda-dad3-48ff-9575-4d42be35ad88 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:02.428807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:02.429821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 04:42:02.430681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:02.432477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 357/710] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:02 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-ba0aabeb-be1c-494c-924f-6004ac8df136 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:02.432587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:02.433481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 04:42:02.434314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:02.438272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae9cda-dad3-48ff-9575-4d42be35ad88 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.439137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 354/711] 10.222.0.105 () {56 vars in 1134 bytes} [Wed Aug 24 04:42:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:02.439762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0aabeb-be1c-494c-924f-6004ac8df136 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.496361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:02.534729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:02.544305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ae9cda-dad3-48ff-9575-4d42be35ad88 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:02.550756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0aabeb-be1c-494c-924f-6004ac8df136 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:02.582015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a155a-439d-4c7f-ba76-270d0b0accd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.582619 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a155a-439d-4c7f-ba76-270d0b0accd2 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.582889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a155a-439d-4c7f-ba76-270d0b0accd2 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.583185 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a155a-439d-4c7f-ba76-270d0b0accd2 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.583861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a155a-439d-4c7f-ba76-270d0b0accd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:02.584158 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a155a-439d-4c7f-ba76-270d0b0accd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.585872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a155a-439d-4c7f-ba76-270d0b0accd2 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.588253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca0939-ab34-4812-99ab-5e122c509852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.588858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca0939-ab34-4812-99ab-5e122c509852 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.589138 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca0939-ab34-4812-99ab-5e122c509852 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.589411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca0939-ab34-4812-99ab-5e122c509852 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.590047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca0939-ab34-4812-99ab-5e122c509852 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:02.590347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca0939-ab34-4812-99ab-5e122c509852 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.591725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca0939-ab34-4812-99ab-5e122c509852 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.598636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 358/712] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:02.599722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a155a-439d-4c7f-ba76-270d0b0accd2 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.611487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 355/713] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:02.612737 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca0939-ab34-4812-99ab-5e122c509852 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.708675 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a155a-439d-4c7f-ba76-270d0b0accd2 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:02.721537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca0939-ab34-4812-99ab-5e122c509852 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:02.724301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:02.751000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce540da-1256-43d6-a280-8efd804a810b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.751635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce540da-1256-43d6-a280-8efd804a810b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.751914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce540da-1256-43d6-a280-8efd804a810b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.752189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce540da-1256-43d6-a280-8efd804a810b None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.752815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce540da-1256-43d6-a280-8efd804a810b None admin] RBAC: Adding query filter params (name=network) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:02.753117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce540da-1256-43d6-a280-8efd804a810b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.754557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce540da-1256-43d6-a280-8efd804a810b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.757847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c8350-2dea-4338-b104-97f8a6b03810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.758438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c8350-2dea-4338-b104-97f8a6b03810 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.758710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c8350-2dea-4338-b104-97f8a6b03810 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.758975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c8350-2dea-4338-b104-97f8a6b03810 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.759614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c8350-2dea-4338-b104-97f8a6b03810 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:02.759909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c8350-2dea-4338-b104-97f8a6b03810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.761238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c8350-2dea-4338-b104-97f8a6b03810 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.764840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 359/714] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:02 2022] GET /identity/v3/services?name=network => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:02.765687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce540da-1256-43d6-a280-8efd804a810b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.775660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 356/715] 10.222.0.105 () {56 vars in 1129 bytes} [Wed Aug 24 04:42:02 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:02.775660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c8350-2dea-4338-b104-97f8a6b03810 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.787858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:02.877140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c8350-2dea-4338-b104-97f8a6b03810 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:02.877474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce540da-1256-43d6-a280-8efd804a810b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:02.911444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a66fe2-7543-471a-a060-967bd2d73f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.912053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a66fe2-7543-471a-a060-967bd2d73f67 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.912328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a66fe2-7543-471a-a060-967bd2d73f67 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.912591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a66fe2-7543-471a-a060-967bd2d73f67 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.913214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a66fe2-7543-471a-a060-967bd2d73f67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.914620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a66fe2-7543-471a-a060-967bd2d73f67 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.914878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fd6ee-caff-4843-8117-940d12f1f241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:02.915514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fd6ee-caff-4843-8117-940d12f1f241 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:02.915790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fd6ee-caff-4843-8117-940d12f1f241 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:02.916062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643fd6ee-caff-4843-8117-940d12f1f241 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:02.916784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fd6ee-caff-4843-8117-940d12f1f241 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=3c822c8f23b74bae84769046fdb85db4) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:02.917191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fd6ee-caff-4843-8117-940d12f1f241 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:02.918607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643fd6ee-caff-4843-8117-940d12f1f241 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:02.924701 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 360/716] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:02 2022] GET /identity/v3/role_assignments?user.id=3c822c8f23b74bae84769046fdb85db4&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:02.925490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fd6ee-caff-4843-8117-940d12f1f241 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:02.975731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:03.023986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 357/717] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:42:02 2022] POST /identity/v3/users => generated 255 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:03.025685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a66fe2-7543-471a-a060-967bd2d73f67 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:03.049595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643fd6ee-caff-4843-8117-940d12f1f241 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:03.112544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a25e0d-da92-4722-8fc4-ded5328915e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:03.113161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a25e0d-da92-4722-8fc4-ded5328915e5 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:03.113440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a25e0d-da92-4722-8fc4-ded5328915e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:03.113713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a25e0d-da92-4722-8fc4-ded5328915e5 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:03.114402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a25e0d-da92-4722-8fc4-ded5328915e5 None admin] RBAC: Adding query filter params (type=network) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:03.114706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a25e0d-da92-4722-8fc4-ded5328915e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:03.116404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a25e0d-da92-4722-8fc4-ded5328915e5 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:03.123324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 361/718] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:02 2022] GET /identity/v3/services?type=network => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:03.134204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:03.181174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a66fe2-7543-471a-a060-967bd2d73f67 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:03.229437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41447248-a5cb-4168-afbe-5b9c2651c849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:03.230080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41447248-a5cb-4168-afbe-5b9c2651c849 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:03.230329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41447248-a5cb-4168-afbe-5b9c2651c849 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:03.230598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41447248-a5cb-4168-afbe-5b9c2651c849 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:03.231252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41447248-a5cb-4168-afbe-5b9c2651c849 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:03.231552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41447248-a5cb-4168-afbe-5b9c2651c849 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:03.232941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41447248-a5cb-4168-afbe-5b9c2651c849 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:03.243510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 358/719] 10.222.0.105 () {56 vars in 1129 bytes} [Wed Aug 24 04:42:03 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:03.952040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b75e2-62d1-4562-a02f-617b12aa3ac7 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:03.953789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b75e2-62d1-4562-a02f-617b12aa3ac7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:03.954280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b75e2-62d1-4562-a02f-617b12aa3ac7 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:03.956095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 362/720] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:03 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:03.961928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20ae64-6aa4-4c71-80b2-ae8de3e16009 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:03.962470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20ae64-6aa4-4c71-80b2-ae8de3e16009 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:03.962959 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20ae64-6aa4-4c71-80b2-ae8de3e16009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:03.987321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6e20ae64-6aa4-4c71-80b2-ae8de3e16009 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:03.988881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6e20ae64-6aa4-4c71-80b2-ae8de3e16009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:04.085287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 359/721] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:42:04.357987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b75e2-62d1-4562-a02f-617b12aa3ac7 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:04.628786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b75e2-62d1-4562-a02f-617b12aa3ac7 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:04.636426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:04.673991 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dafa8-1038-476c-871c-f0a0400534fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:04.674637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dafa8-1038-476c-871c-f0a0400534fa None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:04.674915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dafa8-1038-476c-871c-f0a0400534fa None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:04.675208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dafa8-1038-476c-871c-f0a0400534fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:04.689412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463dafa8-1038-476c-871c-f0a0400534fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-463dafa8-1038-476c-871c-f0a0400534fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:04.694385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:04.695678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:04.696531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:04.700838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 363/722] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:04.702287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20ae64-6aa4-4c71-80b2-ae8de3e16009 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:04.821498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:04.886759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20ae64-6aa4-4c71-80b2-ae8de3e16009 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:04.932665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a62862-b4dc-4dca-9c1b-799627548e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:04.933264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a62862-b4dc-4dca-9c1b-799627548e02 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:04.933538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a62862-b4dc-4dca-9c1b-799627548e02 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:04.933802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a62862-b4dc-4dca-9c1b-799627548e02 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:04.934452 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a62862-b4dc-4dca-9c1b-799627548e02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:04.934745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a62862-b4dc-4dca-9c1b-799627548e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:04.936129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a62862-b4dc-4dca-9c1b-799627548e02 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:04.950613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 360/723] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:42:04 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:04.956225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dafa8-1038-476c-871c-f0a0400534fa None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:05.100925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dafa8-1038-476c-871c-f0a0400534fa None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:05.141415 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c1a8a-a458-452a-a651-356f842f854e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:05.141960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c1a8a-a458-452a-a651-356f842f854e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.142237 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c1a8a-a458-452a-a651-356f842f854e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.142507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9c1a8a-a458-452a-a651-356f842f854e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.157064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c1a8a-a458-452a-a651-356f842f854e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:05.158516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9c1a8a-a458-452a-a651-356f842f854e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-9f9c1a8a-a458-452a-a651-356f842f854e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:05.184539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:05.185577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:05.186473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:05.186473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:05.186473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:05.187801 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 364/724] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:05.196138 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a62862-b4dc-4dca-9c1b-799627548e02 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:05.371649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a62862-b4dc-4dca-9c1b-799627548e02 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:05.407075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:05.423710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90878ed9-2ef4-442b-9e20-e1795e124af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:05.424327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90878ed9-2ef4-442b-9e20-e1795e124af8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.424608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90878ed9-2ef4-442b-9e20-e1795e124af8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.431812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90878ed9-2ef4-442b-9e20-e1795e124af8 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.431812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90878ed9-2ef4-442b-9e20-e1795e124af8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:05.431812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90878ed9-2ef4-442b-9e20-e1795e124af8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:05.431812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90878ed9-2ef4-442b-9e20-e1795e124af8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:05.441456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 361/725] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:05.447574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c1a8a-a458-452a-a651-356f842f854e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:05.483575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:05.488528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926202b2-45ac-4f67-81b0-5f6b4edc96c9 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.488814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926202b2-45ac-4f67-81b0-5f6b4edc96c9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.489096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926202b2-45ac-4f67-81b0-5f6b4edc96c9 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.490690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 362/726] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:05.495576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9c30c-9eb3-4d83-83d1-18350c185073 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.495860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9c30c-9eb3-4d83-83d1-18350c185073 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.496134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be9c30c-9eb3-4d83-83d1-18350c185073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.515049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9be9c30c-9eb3-4d83-83d1-18350c185073 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:05.516543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9be9c30c-9eb3-4d83-83d1-18350c185073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:05.599327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 363/727] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 24 04:42:05.603987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313054d1-a1f1-41cc-8f34-a9334201588c None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.604268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313054d1-a1f1-41cc-8f34-a9334201588c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.604535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313054d1-a1f1-41cc-8f34-a9334201588c None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.606032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 364/728] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:05 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:05.612184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8224e0-d2cb-4539-8701-a791e9a621fa None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.612756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8224e0-d2cb-4539-8701-a791e9a621fa None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.613227 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8224e0-d2cb-4539-8701-a791e9a621fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.634763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ba8224e0-d2cb-4539-8701-a791e9a621fa None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:05.637346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ba8224e0-d2cb-4539-8701-a791e9a621fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:05.651102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9c1a8a-a458-452a-a651-356f842f854e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:05.719672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eaa49d-941a-4f37-bd0b-60a929f1b717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:05.720230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaa49d-941a-4f37-bd0b-60a929f1b717 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.720502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaa49d-941a-4f37-bd0b-60a929f1b717 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.720769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eaa49d-941a-4f37-bd0b-60a929f1b717 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.735869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eaa49d-941a-4f37-bd0b-60a929f1b717 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:05.737320 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eaa49d-941a-4f37-bd0b-60a929f1b717 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-e0eaa49d-941a-4f37-bd0b-60a929f1b717 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:05.764492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:05.765520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:05.766673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 365/729] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:05.770543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 365/730] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:42:05.770996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bee24f-d83f-4170-a8f7-c3d13e028861 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.771313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bee24f-d83f-4170-a8f7-c3d13e028861 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.771585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bee24f-d83f-4170-a8f7-c3d13e028861 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.773085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 366/731] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:05.778186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9572a-53ef-4711-907c-ff1ea9b68402 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.778474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9572a-53ef-4711-907c-ff1ea9b68402 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.778813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9572a-53ef-4711-907c-ff1ea9b68402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.782629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bee24f-d83f-4170-a8f7-c3d13e028861 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:05.811061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ccb9572a-53ef-4711-907c-ff1ea9b68402 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:05.812358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ccb9572a-53ef-4711-907c-ff1ea9b68402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:05.867682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:05.902090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:05.903966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 366/732] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:42:05.905278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e0511-8fc2-45da-bec5-a941bebd913d None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.906657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e0511-8fc2-45da-bec5-a941bebd913d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.907160 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e0511-8fc2-45da-bec5-a941bebd913d None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.912861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 367/733] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:05 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:05.913600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e0511-8fc2-45da-bec5-a941bebd913d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:05.933030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bee24f-d83f-4170-a8f7-c3d13e028861 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:05.971965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68b23a-b81d-4700-921d-bdc0f47b1e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:05.973438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b23a-b81d-4700-921d-bdc0f47b1e4c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.973927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b23a-b81d-4700-921d-bdc0f47b1e4c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.974427 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68b23a-b81d-4700-921d-bdc0f47b1e4c None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:05.975444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68b23a-b81d-4700-921d-bdc0f47b1e4c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:05.976059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68b23a-b81d-4700-921d-bdc0f47b1e4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:05.977679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68b23a-b81d-4700-921d-bdc0f47b1e4c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:05.997465 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 367/734] 10.222.0.105 () {56 vars in 1140 bytes} [Wed Aug 24 04:42:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:05.998934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440d4ac-fbf2-4ee0-9b7c-65c13486c877 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:05.999514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440d4ac-fbf2-4ee0-9b7c-65c13486c877 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:05.999981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440d4ac-fbf2-4ee0-9b7c-65c13486c877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:06.023765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-1440d4ac-fbf2-4ee0-9b7c-65c13486c877 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:06.025498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1440d4ac-fbf2-4ee0-9b7c-65c13486c877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:06.046462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e0511-8fc2-45da-bec5-a941bebd913d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:06.084065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9440f-c443-48c2-b133-118bfd7bcfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:06.084701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9440f-c443-48c2-b133-118bfd7bcfbb None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:06.084982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9440f-c443-48c2-b133-118bfd7bcfbb None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:06.085254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9440f-c443-48c2-b133-118bfd7bcfbb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:06.096431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9440f-c443-48c2-b133-118bfd7bcfbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-53b9440f-c443-48c2-b133-118bfd7bcfbb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:06.101295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:06.102261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 368/735] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:06.107918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9440f-c443-48c2-b133-118bfd7bcfbb None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:06.123248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 368/736] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 04:42:06.123248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440d4ac-fbf2-4ee0-9b7c-65c13486c877 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:06.150353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:06.197510 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:06.228975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9440f-c443-48c2-b133-118bfd7bcfbb None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:06.232243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440d4ac-fbf2-4ee0-9b7c-65c13486c877 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:06.264494 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d06092-12b1-42d1-8fff-719898b51641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:06.265323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d06092-12b1-42d1-8fff-719898b51641 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:06.265503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a37a13-304d-40dd-bc26-9ca4f094af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:06.265618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d06092-12b1-42d1-8fff-719898b51641 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:06.265928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d06092-12b1-42d1-8fff-719898b51641 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:06.266115 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a37a13-304d-40dd-bc26-9ca4f094af42 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:06.266451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a37a13-304d-40dd-bc26-9ca4f094af42 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:06.266664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a37a13-304d-40dd-bc26-9ca4f094af42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:06.276622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a37a13-304d-40dd-bc26-9ca4f094af42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-75a37a13-304d-40dd-bc26-9ca4f094af42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:06.282209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:06.283408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:06.284297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 369/737] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:42:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:06.285074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a37a13-304d-40dd-bc26-9ca4f094af42 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:06.344927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d06092-12b1-42d1-8fff-719898b51641 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:06.346607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d06092-12b1-42d1-8fff-719898b51641 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:06.376777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:06.385315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a37a13-304d-40dd-bc26-9ca4f094af42 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:06.413312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:06.422488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1951d-9d63-497a-87a4-1c769fb03848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:06.423115 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1951d-9d63-497a-87a4-1c769fb03848 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:06.423400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1951d-9d63-497a-87a4-1c769fb03848 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:06.423665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1951d-9d63-497a-87a4-1c769fb03848 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:06.424327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1951d-9d63-497a-87a4-1c769fb03848 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:06.424624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1951d-9d63-497a-87a4-1c769fb03848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:06.425990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1951d-9d63-497a-87a4-1c769fb03848 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:06.436247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 370/738] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:42:06 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:06.436968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1951d-9d63-497a-87a4-1c769fb03848 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:06.453733 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 369/739] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:06 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 04:42:06.454567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d06092-12b1-42d1-8fff-719898b51641 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:06.536853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1951d-9d63-497a-87a4-1c769fb03848 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:06.553340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d06092-12b1-42d1-8fff-719898b51641 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:06.570433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3a330-8f57-41c8-8c38-f6366195a391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:06.571100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3a330-8f57-41c8-8c38-f6366195a391 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:06.571380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3a330-8f57-41c8-8c38-f6366195a391 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:06.571652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3a330-8f57-41c8-8c38-f6366195a391 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:06.572297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3a330-8f57-41c8-8c38-f6366195a391 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:06.573647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3a330-8f57-41c8-8c38-f6366195a391 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:06.581629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:06.586384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ea528-0c46-47cf-a2f1-c7c9b7d4eb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:06.587027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ea528-0c46-47cf-a2f1-c7c9b7d4eb71 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:06.587342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ea528-0c46-47cf-a2f1-c7c9b7d4eb71 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:06.587615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ea528-0c46-47cf-a2f1-c7c9b7d4eb71 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:06.588266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ea528-0c46-47cf-a2f1-c7c9b7d4eb71 None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:06.588566 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ea528-0c46-47cf-a2f1-c7c9b7d4eb71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:06.589915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ea528-0c46-47cf-a2f1-c7c9b7d4eb71 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:06.598672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 370/740] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:42:06 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:06.599468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ea528-0c46-47cf-a2f1-c7c9b7d4eb71 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:06.685555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 371/741] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:42:06 2022] POST /identity/v3/services => generated 252 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:06.686372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3a330-8f57-41c8-8c38-f6366195a391 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:06.697992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ea528-0c46-47cf-a2f1-c7c9b7d4eb71 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:06.744713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba25ee-86ba-49e4-a05a-5062fe5922f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:06.745335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba25ee-86ba-49e4-a05a-5062fe5922f8 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:06.745610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba25ee-86ba-49e4-a05a-5062fe5922f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:06.745875 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba25ee-86ba-49e4-a05a-5062fe5922f8 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:06.746525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba25ee-86ba-49e4-a05a-5062fe5922f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:06.747940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba25ee-86ba-49e4-a05a-5062fe5922f8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:06.749004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:06.804376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3a330-8f57-41c8-8c38-f6366195a391 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:06.840500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff656bd5-cdb3-43ef-b875-a1d1310afb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:06.841138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff656bd5-cdb3-43ef-b875-a1d1310afb75 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:06.841410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff656bd5-cdb3-43ef-b875-a1d1310afb75 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:06.847044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff656bd5-cdb3-43ef-b875-a1d1310afb75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:06.847260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 371/742] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:42:06 2022] POST /identity/v3/services => generated 241 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:06.847260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba25ee-86ba-49e4-a05a-5062fe5922f8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:06.859705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff656bd5-cdb3-43ef-b875-a1d1310afb75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:06.861226 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff656bd5-cdb3-43ef-b875-a1d1310afb75 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-ff656bd5-cdb3-43ef-b875-a1d1310afb75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:06.879077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:06.880157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:06.881380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:06.881380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:06.881380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:06.881380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 372/743] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:06.881380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff656bd5-cdb3-43ef-b875-a1d1310afb75 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:06.920582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:06.988870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba25ee-86ba-49e4-a05a-5062fe5922f8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.021300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:07.024576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff656bd5-cdb3-43ef-b875-a1d1310afb75 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.036034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c014e40-1460-4505-bbb5-7706a9399163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:07.036671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c014e40-1460-4505-bbb5-7706a9399163 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:07.036957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c014e40-1460-4505-bbb5-7706a9399163 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:07.037232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c014e40-1460-4505-bbb5-7706a9399163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:07.049088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c014e40-1460-4505-bbb5-7706a9399163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:07.050292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c014e40-1460-4505-bbb5-7706a9399163 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:07.065741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181f243-6fc2-4979-80bd-3146b3edbf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:07.066665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181f243-6fc2-4979-80bd-3146b3edbf34 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:07.067107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181f243-6fc2-4979-80bd-3146b3edbf34 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-9c014e40-1460-4505-bbb5-7706a9399163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:07.067411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:07.073524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:07.074872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:07.074872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:07.074872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:07.075094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181f243-6fc2-4979-80bd-3146b3edbf34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:07.075175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 372/744] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:07.075175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c014e40-1460-4505-bbb5-7706a9399163 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:07.084141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181f243-6fc2-4979-80bd-3146b3edbf34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:07.086122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181f243-6fc2-4979-80bd-3146b3edbf34 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-2181f243-6fc2-4979-80bd-3146b3edbf34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:07.108052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:07.109124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:07.110964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:07.110964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:07.110964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:07.110964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 373/745] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:07.115259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181f243-6fc2-4979-80bd-3146b3edbf34 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:07.193412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:07.198347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c014e40-1460-4505-bbb5-7706a9399163 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.243178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181f243-6fc2-4979-80bd-3146b3edbf34 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.243383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276f6b5-6ade-4277-aa2c-66274e2d1ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:07.244071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276f6b5-6ade-4277-aa2c-66274e2d1ae7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:07.244441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276f6b5-6ade-4277-aa2c-66274e2d1ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:07.244797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276f6b5-6ade-4277-aa2c-66274e2d1ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:07.245633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276f6b5-6ade-4277-aa2c-66274e2d1ae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:07.246021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276f6b5-6ade-4277-aa2c-66274e2d1ae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:07.247619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276f6b5-6ade-4277-aa2c-66274e2d1ae7 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:07.262005 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 373/746] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:07.262905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276f6b5-6ade-4277-aa2c-66274e2d1ae7 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:07.289955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cd81e-b28b-4672-b061-9bb1ae0d068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:07.291484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cd81e-b28b-4672-b061-9bb1ae0d068c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:07.292223 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cd81e-b28b-4672-b061-9bb1ae0d068c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:07.293060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145cd81e-b28b-4672-b061-9bb1ae0d068c None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:07.294531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145cd81e-b28b-4672-b061-9bb1ae0d068c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:07.295506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145cd81e-b28b-4672-b061-9bb1ae0d068c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:07.297999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145cd81e-b28b-4672-b061-9bb1ae0d068c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:07.312216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 374/747] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:07.313501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cd81e-b28b-4672-b061-9bb1ae0d068c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:07.329987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:07.376802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276f6b5-6ade-4277-aa2c-66274e2d1ae7 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.425104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145cd81e-b28b-4672-b061-9bb1ae0d068c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.425598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c627bc-f603-4c4b-b205-9ef3b1818ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:07.426721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c627bc-f603-4c4b-b205-9ef3b1818ae8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:07.427256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c627bc-f603-4c4b-b205-9ef3b1818ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:07.428598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c627bc-f603-4c4b-b205-9ef3b1818ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:07.429459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c627bc-f603-4c4b-b205-9ef3b1818ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:07.429943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c627bc-f603-4c4b-b205-9ef3b1818ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:07.431613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c627bc-f603-4c4b-b205-9ef3b1818ae8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:07.450492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 374/748] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:07.451715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c627bc-f603-4c4b-b205-9ef3b1818ae8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:07.467593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fbd2b-813b-4269-bbe3-5a147d5ea77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:07.469303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fbd2b-813b-4269-bbe3-5a147d5ea77e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:07.469433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fbd2b-813b-4269-bbe3-5a147d5ea77e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:07.469802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fbd2b-813b-4269-bbe3-5a147d5ea77e None admin] PATH_INFO: `/v3/users/nova` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:07.477166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fbd2b-813b-4269-bbe3-5a147d5ea77e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:07.478684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fbd2b-813b-4269-bbe3-5a147d5ea77e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:07.479786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-748fbd2b-813b-4269-bbe3-5a147d5ea77e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:07.491175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:07.492135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:07.493440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 375/749] 10.222.0.105 () {56 vars in 1098 bytes} [Wed Aug 24 04:42:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:07.494764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fbd2b-813b-4269-bbe3-5a147d5ea77e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:07.574859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:07.595221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c627bc-f603-4c4b-b205-9ef3b1818ae8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.617420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fbd2b-813b-4269-bbe3-5a147d5ea77e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.640874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98752a00-df98-48c6-bb86-c808db89a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:07.642094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98752a00-df98-48c6-bb86-c808db89a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:07.642529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98752a00-df98-48c6-bb86-c808db89a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:07.643126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98752a00-df98-48c6-bb86-c808db89a2a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:07.651573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98752a00-df98-48c6-bb86-c808db89a2a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:07.653188 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98752a00-df98-48c6-bb86-c808db89a2a0 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:07.655082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078a6f4-b39a-48b9-956b-cf60c3b5ccdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:07.656577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078a6f4-b39a-48b9-956b-cf60c3b5ccdb None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:07.657001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078a6f4-b39a-48b9-956b-cf60c3b5ccdb None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:07.657409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078a6f4-b39a-48b9-956b-cf60c3b5ccdb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:07.665489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078a6f4-b39a-48b9-956b-cf60c3b5ccdb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:07.667163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078a6f4-b39a-48b9-956b-cf60c3b5ccdb None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-98752a00-df98-48c6-bb86-c808db89a2a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:07.667310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:07.668341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:07.669311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:07.669311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:07.669311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:07.669311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:07.669311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:07.669311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:07.669311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:07.669311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:07.671837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 375/750] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:07.672869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98752a00-df98-48c6-bb86-c808db89a2a0 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-4078a6f4-b39a-48b9-956b-cf60c3b5ccdb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:07.681168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:07.682131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:07.683555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:07.683555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:07.683555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:07.683555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:07.683555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:07.683555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:07.683555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:07.683555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:07.683555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:07.683555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:07.687507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 376/751] 10.222.0.105 () {56 vars in 1122 bytes} [Wed Aug 24 04:42:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:07.688501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078a6f4-b39a-48b9-956b-cf60c3b5ccdb None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:07.763137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:07.786925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98752a00-df98-48c6-bb86-c808db89a2a0 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.799243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078a6f4-b39a-48b9-956b-cf60c3b5ccdb None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.824643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86be22-e8f7-4644-8772-9140dcb290cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:07.825691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86be22-e8f7-4644-8772-9140dcb290cf None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:07.826175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:07.828016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86be22-e8f7-4644-8772-9140dcb290cf None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:07.828520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86be22-e8f7-4644-8772-9140dcb290cf None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:07.829643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86be22-e8f7-4644-8772-9140dcb290cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:07.830246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86be22-e8f7-4644-8772-9140dcb290cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:07.832023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2ead6-0802-47c2-b2e3-f29535e9904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:07.832294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d86be22-e8f7-4644-8772-9140dcb290cf None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:07.832589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2ead6-0802-47c2-b2e3-f29535e9904d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:07.832864 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2ead6-0802-47c2-b2e3-f29535e9904d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:07.833136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac2ead6-0802-47c2-b2e3-f29535e9904d None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:07.833814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2ead6-0802-47c2-b2e3-f29535e9904d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:07.834111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2ead6-0802-47c2-b2e3-f29535e9904d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:07.835612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac2ead6-0802-47c2-b2e3-f29535e9904d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:07.850745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 377/752] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:07.851730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2ead6-0802-47c2-b2e3-f29535e9904d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:07.854516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 376/753] 10.222.0.105 () {56 vars in 1138 bytes} [Wed Aug 24 04:42:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:07.861635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86be22-e8f7-4644-8772-9140dcb290cf None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:07.963684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac2ead6-0802-47c2-b2e3-f29535e9904d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:07.991473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d86be22-e8f7-4644-8772-9140dcb290cf None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:08.002540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292410cf-caff-4c35-ab43-a5257cf0d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:08.003279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292410cf-caff-4c35-ab43-a5257cf0d999 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:08.003554 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292410cf-caff-4c35-ab43-a5257cf0d999 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:08.003819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292410cf-caff-4c35-ab43-a5257cf0d999 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:08.004661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292410cf-caff-4c35-ab43-a5257cf0d999 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:08.004962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292410cf-caff-4c35-ab43-a5257cf0d999 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:08.006386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292410cf-caff-4c35-ab43-a5257cf0d999 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:08.027624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 378/754] 10.222.0.105 () {56 vars in 1154 bytes} [Wed Aug 24 04:42:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:08.028475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292410cf-caff-4c35-ab43-a5257cf0d999 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:08.037808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17dc7d-a4ae-4806-90ec-8336349c63c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:08.038444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17dc7d-a4ae-4806-90ec-8336349c63c8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:08.038715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17dc7d-a4ae-4806-90ec-8336349c63c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:08.038980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17dc7d-a4ae-4806-90ec-8336349c63c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:08.042063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:08.051101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17dc7d-a4ae-4806-90ec-8336349c63c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:08.052809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c17dc7d-a4ae-4806-90ec-8336349c63c8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-5c17dc7d-a4ae-4806-90ec-8336349c63c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:08.067646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:08.068625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:08.069805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:08.069805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:08.069805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:08.071915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 377/755] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:08.072911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17dc7d-a4ae-4806-90ec-8336349c63c8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:08.098553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:08.150393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292410cf-caff-4c35-ab43-a5257cf0d999 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:08.195482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c17dc7d-a4ae-4806-90ec-8336349c63c8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:08.205759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782b186-41ea-4dd4-b20e-0ba0dd0a39ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:08.206669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b186-41ea-4dd4-b20e-0ba0dd0a39ea None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:08.207645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b186-41ea-4dd4-b20e-0ba0dd0a39ea None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:08.208094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782b186-41ea-4dd4-b20e-0ba0dd0a39ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:08.209636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782b186-41ea-4dd4-b20e-0ba0dd0a39ea None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=e6a894e4e6f24febbd8397fb8c4c7fb4) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:08.210221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782b186-41ea-4dd4-b20e-0ba0dd0a39ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:08.212542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782b186-41ea-4dd4-b20e-0ba0dd0a39ea None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:08.220866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 379/756] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:08 2022] GET /identity/v3/role_assignments?user.id=e6a894e4e6f24febbd8397fb8c4c7fb4&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:08.221858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782b186-41ea-4dd4-b20e-0ba0dd0a39ea None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:08.265036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462b5c1-3b74-4f2a-b38f-682992fd4b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:08.265669 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462b5c1-3b74-4f2a-b38f-682992fd4b07 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:08.265941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462b5c1-3b74-4f2a-b38f-682992fd4b07 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:08.266213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5462b5c1-3b74-4f2a-b38f-682992fd4b07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:08.367959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782b186-41ea-4dd4-b20e-0ba0dd0a39ea None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:08.384622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:08.408855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:08.412958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18070466-6a23-4165-bc0a-727e914c89e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:08.414031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18070466-6a23-4165-bc0a-727e914c89e0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:08.414846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18070466-6a23-4165-bc0a-727e914c89e0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:08.415544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18070466-6a23-4165-bc0a-727e914c89e0 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:08.416453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18070466-6a23-4165-bc0a-727e914c89e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:08.416898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18070466-6a23-4165-bc0a-727e914c89e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:08.418554 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18070466-6a23-4165-bc0a-727e914c89e0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:08.423312 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462b5c1-3b74-4f2a-b38f-682992fd4b07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:08.424153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5462b5c1-3b74-4f2a-b38f-682992fd4b07 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:08.438889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 380/757] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:08.444193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18070466-6a23-4165-bc0a-727e914c89e0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:08.564758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18070466-6a23-4165-bc0a-727e914c89e0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:08.604307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84dad5-9251-4d23-8649-74314edf830c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:08.605294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84dad5-9251-4d23-8649-74314edf830c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:08.605748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84dad5-9251-4d23-8649-74314edf830c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:08.606190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84dad5-9251-4d23-8649-74314edf830c None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:08.614614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84dad5-9251-4d23-8649-74314edf830c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:08.616744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84dad5-9251-4d23-8649-74314edf830c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-6c84dad5-9251-4d23-8649-74314edf830c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:08.629417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:08.630314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:08.631250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:08.631250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:08.631250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:08.631250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:08.631250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:08.631250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:08.631250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:08.631250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:08.633633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 381/758] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:08.639184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84dad5-9251-4d23-8649-74314edf830c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:08.661578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 378/759] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:08 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:42:08.666818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462b5c1-3b74-4f2a-b38f-682992fd4b07 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:08.712182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:08.790858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84dad5-9251-4d23-8649-74314edf830c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:08.833590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081a118-fb6d-4693-ac66-9fae9536ec0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:08.834613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081a118-fb6d-4693-ac66-9fae9536ec0f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:08.835109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081a118-fb6d-4693-ac66-9fae9536ec0f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:08.835595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081a118-fb6d-4693-ac66-9fae9536ec0f None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:08.836471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081a118-fb6d-4693-ac66-9fae9536ec0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:08.836774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081a118-fb6d-4693-ac66-9fae9536ec0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:08.838574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081a118-fb6d-4693-ac66-9fae9536ec0f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:08.851329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:08.855815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 382/760] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:08.857358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d676217-7a65-426e-b4c8-f3918d5d3013 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:08.857641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d676217-7a65-426e-b4c8-f3918d5d3013 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:08.858434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d676217-7a65-426e-b4c8-f3918d5d3013 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:08.864897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 383/761] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:08 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:08.864897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d676217-7a65-426e-b4c8-f3918d5d3013 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:08.870568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5462b5c1-3b74-4f2a-b38f-682992fd4b07 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:08.975118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:08.982618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d676217-7a65-426e-b4c8-f3918d5d3013 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:09.021976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32719f8-a1b7-4193-8c2d-01ad13b5481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:09.022652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32719f8-a1b7-4193-8c2d-01ad13b5481e None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:09.022918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32719f8-a1b7-4193-8c2d-01ad13b5481e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:09.023233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32719f8-a1b7-4193-8c2d-01ad13b5481e None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/3c822c8f23b74bae84769046fdb85db4/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:09.025184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32719f8-a1b7-4193-8c2d-01ad13b5481e None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=3c822c8f23b74bae84769046fdb85db4, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:09.026611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32719f8-a1b7-4193-8c2d-01ad13b5481e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:09.027225 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3691240-dca5-48b0-b5b5-4d0811df2b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:09.028085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3691240-dca5-48b0-b5b5-4d0811df2b89 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:09.028564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3691240-dca5-48b0-b5b5-4d0811df2b89 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:09.028990 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3691240-dca5-48b0-b5b5-4d0811df2b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:09.040057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3691240-dca5-48b0-b5b5-4d0811df2b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:09.041571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3691240-dca5-48b0-b5b5-4d0811df2b89 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-c3691240-dca5-48b0-b5b5-4d0811df2b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:09.056910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:09.064045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:09.064969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:09.064969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:09.064969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:09.064969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 379/762] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:09.064969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008d48-0757-4c55-b1f6-2e05bb6409c5 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:09.064969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008d48-0757-4c55-b1f6-2e05bb6409c5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:09.064969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73008d48-0757-4c55-b1f6-2e05bb6409c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:09.080615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-73008d48-0757-4c55-b1f6-2e05bb6409c5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:09.081808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-73008d48-0757-4c55-b1f6-2e05bb6409c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:09.134446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 384/763] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:42:08 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/3c822c8f23b74bae84769046fdb85db4/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:42:09.136315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f373ad-536f-4865-aa38-007d75f50e40 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:09.136603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f373ad-536f-4865-aa38-007d75f50e40 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:09.136877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f373ad-536f-4865-aa38-007d75f50e40 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:09.138322 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 385/764] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:09.138893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f373ad-536f-4865-aa38-007d75f50e40 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:09.167071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 380/765] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:09 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:09.167071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a1ff1-023c-42e0-96e5-74d8b2ed549d None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:09.167071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a1ff1-023c-42e0-96e5-74d8b2ed549d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:09.167071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a1ff1-023c-42e0-96e5-74d8b2ed549d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:09.208920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-df1a1ff1-023c-42e0-96e5-74d8b2ed549d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:09.210145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-df1a1ff1-023c-42e0-96e5-74d8b2ed549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:09.323120 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:09.330404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 381/766] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:09 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:09.340156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f373ad-536f-4865-aa38-007d75f50e40 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:09.376196 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31a32d-a326-434f-816d-376e75551969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:09.377143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31a32d-a326-434f-816d-376e75551969 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:09.377644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31a32d-a326-434f-816d-376e75551969 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:09.378118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31a32d-a326-434f-816d-376e75551969 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:09.378951 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31a32d-a326-434f-816d-376e75551969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:09.379600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31a32d-a326-434f-816d-376e75551969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:09.381241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31a32d-a326-434f-816d-376e75551969 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:09.394224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 386/767] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:09.399723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a1ff1-023c-42e0-96e5-74d8b2ed549d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:09.483866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31a32d-a326-434f-816d-376e75551969 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:09.505350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:09.513838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a1ff1-023c-42e0-96e5-74d8b2ed549d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:09.598222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff390ca1-2e74-4078-a664-4b058023c854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:09.598954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff390ca1-2e74-4078-a664-4b058023c854 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:09.599185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff390ca1-2e74-4078-a664-4b058023c854 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:09.599458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff390ca1-2e74-4078-a664-4b058023c854 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:09.608731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31a32d-a326-434f-816d-376e75551969 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:09.617230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff390ca1-2e74-4078-a664-4b058023c854 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:09.618488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff390ca1-2e74-4078-a664-4b058023c854 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-ff390ca1-2e74-4078-a664-4b058023c854 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:09.643829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:09.644772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:09.645675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:09.645675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:09.645675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:09.645675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:09.645675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:09.645675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:09.645675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:09.645675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:09.652201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 382/768] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:09.659004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff390ca1-2e74-4078-a664-4b058023c854 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:09.674349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86d84b-b3cb-4949-b5fe-ea1f0add9de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:09.674998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86d84b-b3cb-4949-b5fe-ea1f0add9de0 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:09.675298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86d84b-b3cb-4949-b5fe-ea1f0add9de0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:09.675569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86d84b-b3cb-4949-b5fe-ea1f0add9de0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:09.676242 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86d84b-b3cb-4949-b5fe-ea1f0add9de0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:09.677706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86d84b-b3cb-4949-b5fe-ea1f0add9de0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-cf86d84b-b3cb-4949-b5fe-ea1f0add9de0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:09.697666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:09.698669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 04:42:09.699539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:09.703892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 387/769] 10.222.0.105 () {56 vars in 1134 bytes} [Wed Aug 24 04:42:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:09.709772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86d84b-b3cb-4949-b5fe-ea1f0add9de0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:09.739120 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:09.776069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff390ca1-2e74-4078-a664-4b058023c854 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:09.814612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:09.824159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f9f80-6b21-40bc-9378-275b93f3b1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:09.825009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f9f80-6b21-40bc-9378-275b93f3b1bb None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:09.825903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f9f80-6b21-40bc-9378-275b93f3b1bb None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:09.826891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f9f80-6b21-40bc-9378-275b93f3b1bb None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:09.828048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f9f80-6b21-40bc-9378-275b93f3b1bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:09.828534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f9f80-6b21-40bc-9378-275b93f3b1bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:09.830769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f9f80-6b21-40bc-9378-275b93f3b1bb None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:09.838616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86d84b-b3cb-4949-b5fe-ea1f0add9de0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:09.846911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 383/770] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:09.848001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f9f80-6b21-40bc-9378-275b93f3b1bb None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:09.884437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569735a2-f020-4b66-aada-78cdf7715f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:09.885595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569735a2-f020-4b66-aada-78cdf7715f03 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:09.885976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569735a2-f020-4b66-aada-78cdf7715f03 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:09.886613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569735a2-f020-4b66-aada-78cdf7715f03 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:09.887395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569735a2-f020-4b66-aada-78cdf7715f03 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:09.888037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569735a2-f020-4b66-aada-78cdf7715f03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:09.890174 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569735a2-f020-4b66-aada-78cdf7715f03 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:09.899093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 388/771] 10.222.0.105 () {56 vars in 1129 bytes} [Wed Aug 24 04:42:09 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:09.900122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569735a2-f020-4b66-aada-78cdf7715f03 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:09.977933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f9f80-6b21-40bc-9378-275b93f3b1bb None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:10.003733 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:10.020663 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdc4f5-b9d9-42bd-a765-a0edacb0a0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:10.021715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdc4f5-b9d9-42bd-a765-a0edacb0a0a0 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:10.021998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdc4f5-b9d9-42bd-a765-a0edacb0a0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:10.022263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdc4f5-b9d9-42bd-a765-a0edacb0a0a0 None admin] PATH_INFO: `/v3/services/network` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:10.022838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdc4f5-b9d9-42bd-a765-a0edacb0a0a0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:10.024492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdc4f5-b9d9-42bd-a765-a0edacb0a0a0 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-21fdc4f5-b9d9-42bd-a765-a0edacb0a0a0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:10.035967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:10.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 04:42:10.037902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:10.039259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569735a2-f020-4b66-aada-78cdf7715f03 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:10.042148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 384/772] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:09 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:10.043924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdc4f5-b9d9-42bd-a765-a0edacb0a0a0 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:10.087312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8093b39-be38-4c3b-9a81-ad26ea8fdf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:10.089116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8093b39-be38-4c3b-9a81-ad26ea8fdf5e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:10.089651 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8093b39-be38-4c3b-9a81-ad26ea8fdf5e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:10.090169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8093b39-be38-4c3b-9a81-ad26ea8fdf5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:10.091120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8093b39-be38-4c3b-9a81-ad26ea8fdf5e None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=da66e763177c47e39b3475fc42ccc8f0) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:10.091672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8093b39-be38-4c3b-9a81-ad26ea8fdf5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:10.094399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8093b39-be38-4c3b-9a81-ad26ea8fdf5e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:10.107120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 389/773] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:09 2022] GET /identity/v3/role_assignments?user.id=da66e763177c47e39b3475fc42ccc8f0&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:10.107120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8093b39-be38-4c3b-9a81-ad26ea8fdf5e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:10.123767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:10.228216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdc4f5-b9d9-42bd-a765-a0edacb0a0a0 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:10.235178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8093b39-be38-4c3b-9a81-ad26ea8fdf5e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:10.272854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5c226-e884-4a00-a152-623f8513f93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:10.273480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5c226-e884-4a00-a152-623f8513f93b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:10.273761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5c226-e884-4a00-a152-623f8513f93b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:10.274082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5c226-e884-4a00-a152-623f8513f93b None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:10.274716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5c226-e884-4a00-a152-623f8513f93b None admin] RBAC: Adding query filter params (name=network) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:10.275033 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5c226-e884-4a00-a152-623f8513f93b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:10.276401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5c226-e884-4a00-a152-623f8513f93b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:10.281471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 390/774] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:10 2022] GET /identity/v3/services?name=network => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:10.286899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26610523-882b-4327-bfad-c3a9a2f0c66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:10.287569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26610523-882b-4327-bfad-c3a9a2f0c66b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:10.287848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26610523-882b-4327-bfad-c3a9a2f0c66b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:10.288111 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26610523-882b-4327-bfad-c3a9a2f0c66b None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:10.288752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26610523-882b-4327-bfad-c3a9a2f0c66b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:10.289052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26610523-882b-4327-bfad-c3a9a2f0c66b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:10.290449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26610523-882b-4327-bfad-c3a9a2f0c66b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:10.292153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5c226-e884-4a00-a152-623f8513f93b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:10.297552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 385/775] 10.222.0.105 () {56 vars in 1129 bytes} [Wed Aug 24 04:42:10 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:10.303333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26610523-882b-4327-bfad-c3a9a2f0c66b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:10.360830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:10.401352 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:10.421988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26610523-882b-4327-bfad-c3a9a2f0c66b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:10.444899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5c226-e884-4a00-a152-623f8513f93b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:10.464036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260429a-f3ba-47ce-90f4-f59270393492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:10.464750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260429a-f3ba-47ce-90f4-f59270393492 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:10.465032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260429a-f3ba-47ce-90f4-f59270393492 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:10.465300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260429a-f3ba-47ce-90f4-f59270393492 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:10.465996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260429a-f3ba-47ce-90f4-f59270393492 None admin] RBAC: Adding query filter params (interface=public, service_id=fd6fdd33f26f487db4fbc31eee69c2e9, region_id=RegionOne) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:10.466297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260429a-f3ba-47ce-90f4-f59270393492 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:10.468383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260429a-f3ba-47ce-90f4-f59270393492 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:10.476877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 386/776] 10.222.0.105 () {56 vars in 1256 bytes} [Wed Aug 24 04:42:10 2022] GET /identity/v3/endpoints?service_id=fd6fdd33f26f487db4fbc31eee69c2e9&interface=public®ion_id=RegionOne => generated 195 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:10.483169 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a7e57-71e7-4428-b0bb-1af86c3e550d None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:10.483473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a7e57-71e7-4428-b0bb-1af86c3e550d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:10.483752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a7e57-71e7-4428-b0bb-1af86c3e550d None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:10.485283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 387/777] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:10 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:10.486019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a7e57-71e7-4428-b0bb-1af86c3e550d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:10.492816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ed1773-cce6-4e00-a975-05e9e8bd6708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:10.493442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed1773-cce6-4e00-a975-05e9e8bd6708 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:10.493719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed1773-cce6-4e00-a975-05e9e8bd6708 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:10.493990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ed1773-cce6-4e00-a975-05e9e8bd6708 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:10.494629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed1773-cce6-4e00-a975-05e9e8bd6708 None admin] RBAC: Adding query filter params (type=network) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:10.494926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed1773-cce6-4e00-a975-05e9e8bd6708 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:10.496326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ed1773-cce6-4e00-a975-05e9e8bd6708 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:10.502982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 391/778] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:10 2022] GET /identity/v3/services?type=network => generated 352 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:10.508890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6fd85-40df-40b0-906d-e9234e2f8938 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:10.509355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6fd85-40df-40b0-906d-e9234e2f8938 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:10.509634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6fd85-40df-40b0-906d-e9234e2f8938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:10.528350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-3ab6fd85-40df-40b0-906d-e9234e2f8938 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:10.530175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3ab6fd85-40df-40b0-906d-e9234e2f8938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:10.602317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 392/779] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:10 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:10.603289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6fd85-40df-40b0-906d-e9234e2f8938 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:10.627405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a7e57-71e7-4428-b0bb-1af86c3e550d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:10.649556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:10.692193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be570a-6a2d-4aeb-a6e5-01d1a6471214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:10.692804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be570a-6a2d-4aeb-a6e5-01d1a6471214 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:10.693079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be570a-6a2d-4aeb-a6e5-01d1a6471214 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:10.693602 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be570a-6a2d-4aeb-a6e5-01d1a6471214 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:10.694453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be570a-6a2d-4aeb-a6e5-01d1a6471214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:10.695903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be570a-6a2d-4aeb-a6e5-01d1a6471214 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:10.705095 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 388/780] 10.222.0.105 () {56 vars in 1092 bytes} [Wed Aug 24 04:42:10 2022] GET /identity/v3/services => generated 776 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:10.736378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:10.812691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6fd85-40df-40b0-906d-e9234e2f8938 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:10.852040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ba046-6b3e-4924-b711-3b0ee30f58e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:10.852668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ba046-6b3e-4924-b711-3b0ee30f58e4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:10.852948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ba046-6b3e-4924-b711-3b0ee30f58e4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:10.853279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ba046-6b3e-4924-b711-3b0ee30f58e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:10.853993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ba046-6b3e-4924-b711-3b0ee30f58e4 None admin] RBAC: Adding query filter params (interface=public, service_id=f34e268353a84173a5d3e0e7d54d3035, region_id=RegionOne) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:10.854363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ba046-6b3e-4924-b711-3b0ee30f58e4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:10.855773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ba046-6b3e-4924-b711-3b0ee30f58e4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:10.865739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 393/781] 10.222.0.105 () {56 vars in 1256 bytes} [Wed Aug 24 04:42:10 2022] GET /identity/v3/endpoints?service_id=f34e268353a84173a5d3e0e7d54d3035&interface=public®ion_id=RegionOne => generated 195 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:10.870925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be570a-6a2d-4aeb-a6e5-01d1a6471214 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:10.972572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ba046-6b3e-4924-b711-3b0ee30f58e4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:11.033965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be570a-6a2d-4aeb-a6e5-01d1a6471214 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:11.071773 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f776e-0801-4eb6-ba71-e853ad014042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:11.072455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f776e-0801-4eb6-ba71-e853ad014042 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:11.072742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f776e-0801-4eb6-ba71-e853ad014042 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:11.073017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f776e-0801-4eb6-ba71-e853ad014042 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:11.073682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f776e-0801-4eb6-ba71-e853ad014042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:11.075132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5f776e-0801-4eb6-ba71-e853ad014042 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:11.080787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 389/782] 10.222.0.105 () {56 vars in 1092 bytes} [Wed Aug 24 04:42:10 2022] GET /identity/v3/services => generated 776 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:11.141398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ba046-6b3e-4924-b711-3b0ee30f58e4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:11.173536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:11.182821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31606dab-6853-47b2-84f7-51f0929dd141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:11.183498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31606dab-6853-47b2-84f7-51f0929dd141 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:11.183777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31606dab-6853-47b2-84f7-51f0929dd141 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:11.184825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31606dab-6853-47b2-84f7-51f0929dd141 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:11.209209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31606dab-6853-47b2-84f7-51f0929dd141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-31606dab-6853-47b2-84f7-51f0929dd141 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:11.214477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:11.215724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:11.216884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:11.221253 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 394/783] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:11.229238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f776e-0801-4eb6-ba71-e853ad014042 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:11.309447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:11.367700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f776e-0801-4eb6-ba71-e853ad014042 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:11.402966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bd851-7ade-48c3-8a7c-26b8666436d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:11.403609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bd851-7ade-48c3-8a7c-26b8666436d2 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:11.403889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bd851-7ade-48c3-8a7c-26b8666436d2 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:11.404829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bd851-7ade-48c3-8a7c-26b8666436d2 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:11.405473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bd851-7ade-48c3-8a7c-26b8666436d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:11.405771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bd851-7ade-48c3-8a7c-26b8666436d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:11.407151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bd851-7ade-48c3-8a7c-26b8666436d2 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:11.416186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 390/784] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:42:11 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:11.421917 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31606dab-6853-47b2-84f7-51f0929dd141 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:11.494255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c98e1a-a21d-4e6b-bcac-600a23408185 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:11.494445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c98e1a-a21d-4e6b-bcac-600a23408185 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:11.494719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c98e1a-a21d-4e6b-bcac-600a23408185 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:11.496478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 391/785] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:11.502645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860195c1-a9ac-43bb-aad5-61d867ae3047 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:11.502923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860195c1-a9ac-43bb-aad5-61d867ae3047 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:11.503246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860195c1-a9ac-43bb-aad5-61d867ae3047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:11.521348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-860195c1-a9ac-43bb-aad5-61d867ae3047 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:11.522586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-860195c1-a9ac-43bb-aad5-61d867ae3047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:11.549891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31606dab-6853-47b2-84f7-51f0929dd141 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:11.595345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cf013-c653-4415-bc48-9b3b88ff2366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:11.596372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cf013-c653-4415-bc48-9b3b88ff2366 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:11.596854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cf013-c653-4415-bc48-9b3b88ff2366 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:11.597423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67cf013-c653-4415-bc48-9b3b88ff2366 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:11.616282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 392/786] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:11 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:11.617566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67cf013-c653-4415-bc48-9b3b88ff2366 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:11.619257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67cf013-c653-4415-bc48-9b3b88ff2366 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-a67cf013-c653-4415-bc48-9b3b88ff2366 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:11.636324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:11.637365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:11.638278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:11.638278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:11.638278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:11.641149 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 395/787] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:11.648149 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860195c1-a9ac-43bb-aad5-61d867ae3047 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:11.695289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:11.793348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860195c1-a9ac-43bb-aad5-61d867ae3047 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:11.837955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67c578-a2f3-461f-b96e-e52935f968cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:11.838922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67c578-a2f3-461f-b96e-e52935f968cd None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:11.839425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67c578-a2f3-461f-b96e-e52935f968cd None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:11.839893 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea67c578-a2f3-461f-b96e-e52935f968cd None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:11.840728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67c578-a2f3-461f-b96e-e52935f968cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:11.841250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67c578-a2f3-461f-b96e-e52935f968cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:11.843446 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea67c578-a2f3-461f-b96e-e52935f968cd None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:11.861806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 393/788] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:11.867291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cf013-c653-4415-bc48-9b3b88ff2366 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:11.923594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:11.992010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67c578-a2f3-461f-b96e-e52935f968cd None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:12.003362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67cf013-c653-4415-bc48-9b3b88ff2366 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:12.063552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a8da4-087a-436a-abf2-1d074eb8fba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:12.064542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a8da4-087a-436a-abf2-1d074eb8fba1 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:12.065045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a8da4-087a-436a-abf2-1d074eb8fba1 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:12.065546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a8da4-087a-436a-abf2-1d074eb8fba1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:12.074002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a8da4-087a-436a-abf2-1d074eb8fba1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:12.076254 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a8da4-087a-436a-abf2-1d074eb8fba1 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-670a8da4-087a-436a-abf2-1d074eb8fba1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:12.088011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:12.089051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:12.090015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:12.090015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:12.090015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:12.090015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:12.090015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:12.090015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:12.090015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:12.090015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:12.090015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:42:12.090015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:12.093277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 396/789] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:12.103306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a8da4-087a-436a-abf2-1d074eb8fba1 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:12.224246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:12.239718 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea67c578-a2f3-461f-b96e-e52935f968cd None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:12.285657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bfd5a-d434-4f21-9c7a-8bd342f464c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:12.286306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bfd5a-d434-4f21-9c7a-8bd342f464c7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:12.286550 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bfd5a-d434-4f21-9c7a-8bd342f464c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:12.286811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bfd5a-d434-4f21-9c7a-8bd342f464c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:12.306903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bfd5a-d434-4f21-9c7a-8bd342f464c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:12.311422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-0f7bfd5a-d434-4f21-9c7a-8bd342f464c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:12.313754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:12.314673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:12.315649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 394/790] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:12.316819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a8da4-087a-436a-abf2-1d074eb8fba1 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:12.320887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bfd5a-d434-4f21-9c7a-8bd342f464c7 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:12.392025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d01f7-dc6b-4cf6-8fb4-f767343e925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:12.392954 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d01f7-dc6b-4cf6-8fb4-f767343e925c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:12.393417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d01f7-dc6b-4cf6-8fb4-f767343e925c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:12.393870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d01f7-dc6b-4cf6-8fb4-f767343e925c None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:12.394789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d01f7-dc6b-4cf6-8fb4-f767343e925c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:12.395328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d01f7-dc6b-4cf6-8fb4-f767343e925c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:12.396934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d01f7-dc6b-4cf6-8fb4-f767343e925c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:12.438374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 397/791] 10.222.0.105 () {56 vars in 1140 bytes} [Wed Aug 24 04:42:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:12.444243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d01f7-dc6b-4cf6-8fb4-f767343e925c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:12.488926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bfd5a-d434-4f21-9c7a-8bd342f464c7 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:12.531532 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3715e-7e2a-4e96-8571-55dea6cb35a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:12.532142 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3715e-7e2a-4e96-8571-55dea6cb35a1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:12.532415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3715e-7e2a-4e96-8571-55dea6cb35a1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:12.532684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3715e-7e2a-4e96-8571-55dea6cb35a1 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:12.533619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3715e-7e2a-4e96-8571-55dea6cb35a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:12.533920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3715e-7e2a-4e96-8571-55dea6cb35a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:12.535874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3715e-7e2a-4e96-8571-55dea6cb35a1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:12.547429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 395/792] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:42:12 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:12.549786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812822-375c-4aec-a1db-b02b7aaa7c46 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:12.550086 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812822-375c-4aec-a1db-b02b7aaa7c46 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:12.550361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91812822-375c-4aec-a1db-b02b7aaa7c46 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:12.555099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 396/793] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:12 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:12.556083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91812822-375c-4aec-a1db-b02b7aaa7c46 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:12.578416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:12.590254 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d01f7-dc6b-4cf6-8fb4-f767343e925c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:12.635627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304a865-0245-468d-9ce4-02a55c14f5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:12.636257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304a865-0245-468d-9ce4-02a55c14f5d5 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:12.636533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304a865-0245-468d-9ce4-02a55c14f5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:12.636812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304a865-0245-468d-9ce4-02a55c14f5d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:12.648379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304a865-0245-468d-9ce4-02a55c14f5d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:12.649566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304a865-0245-468d-9ce4-02a55c14f5d5 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-0304a865-0245-468d-9ce4-02a55c14f5d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:12.665291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:12.666272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:12.667198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:12.667198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:12.667198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:12.667198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 398/794] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:12.668213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cc4c5-bda8-4423-9582-ccb682d7894e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:12.668475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cc4c5-bda8-4423-9582-ccb682d7894e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:12.668753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cc4c5-bda8-4423-9582-ccb682d7894e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:12.687709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91812822-375c-4aec-a1db-b02b7aaa7c46 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:12.688043 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-aa9cc4c5-bda8-4423-9582-ccb682d7894e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:12.689226 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-aa9cc4c5-bda8-4423-9582-ccb682d7894e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:12.693791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:12.735671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7377e1d-8c87-4a0c-80ef-ba17f2044567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:12.736294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377e1d-8c87-4a0c-80ef-ba17f2044567 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:12.736623 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377e1d-8c87-4a0c-80ef-ba17f2044567 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:12.736896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377e1d-8c87-4a0c-80ef-ba17f2044567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:12.749554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7377e1d-8c87-4a0c-80ef-ba17f2044567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:12.751123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7377e1d-8c87-4a0c-80ef-ba17f2044567 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-e7377e1d-8c87-4a0c-80ef-ba17f2044567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:12.768026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:12.769034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:12.769919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:12.769919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:12.769919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:12.769919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 397/795] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:12.770575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7377e1d-8c87-4a0c-80ef-ba17f2044567 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:12.784211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 399/796] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:12 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:12.785797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cc4c5-bda8-4423-9582-ccb682d7894e None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:12.894997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:12.944333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cc4c5-bda8-4423-9582-ccb682d7894e None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:12.945429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7377e1d-8c87-4a0c-80ef-ba17f2044567 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:12.985350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25edf8-9b75-4340-a3b0-468e40060a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:12.985981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25edf8-9b75-4340-a3b0-468e40060a50 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:12.986251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25edf8-9b75-4340-a3b0-468e40060a50 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:12.986527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25edf8-9b75-4340-a3b0-468e40060a50 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:12.987212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25edf8-9b75-4340-a3b0-468e40060a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:12.987514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25edf8-9b75-4340-a3b0-468e40060a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:12.989063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f25edf8-9b75-4340-a3b0-468e40060a50 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:12.990733 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5ecf7-8c32-4fad-b264-49a24bd56d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:12.992107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5ecf7-8c32-4fad-b264-49a24bd56d4c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:12.992598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5ecf7-8c32-4fad-b264-49a24bd56d4c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:12.993061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5ecf7-8c32-4fad-b264-49a24bd56d4c None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:12.993915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5ecf7-8c32-4fad-b264-49a24bd56d4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:12.994406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5ecf7-8c32-4fad-b264-49a24bd56d4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:12.996590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5ecf7-8c32-4fad-b264-49a24bd56d4c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:12.998841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:13.003483 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 400/797] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:13.010991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 398/798] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:13.014544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25edf8-9b75-4340-a3b0-468e40060a50 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:13.018021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5ecf7-8c32-4fad-b264-49a24bd56d4c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:13.195724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f25edf8-9b75-4340-a3b0-468e40060a50 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:13.204700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5ecf7-8c32-4fad-b264-49a24bd56d4c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:13.233008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d0306-2a8e-436a-ab80-69a92594a2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:13.233323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:13.233962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d0306-2a8e-436a-ab80-69a92594a2ed None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.234237 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d0306-2a8e-436a-ab80-69a92594a2ed None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.234497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d0306-2a8e-436a-ab80-69a92594a2ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.241979 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d0306-2a8e-436a-ab80-69a92594a2ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:13.243313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bf2d6-11f0-4bb0-a6d2-5531c0ce0f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:13.248429 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d0306-2a8e-436a-ab80-69a92594a2ed None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:13.248588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bf2d6-11f0-4bb0-a6d2-5531c0ce0f49 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.248588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bf2d6-11f0-4bb0-a6d2-5531c0ce0f49 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.248588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bf2d6-11f0-4bb0-a6d2-5531c0ce0f49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.253480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bf2d6-11f0-4bb0-a6d2-5531c0ce0f49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:13.255088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196bf2d6-11f0-4bb0-a6d2-5531c0ce0f49 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-034d0306-2a8e-436a-ab80-69a92594a2ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:13.255286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:13.256182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 401/799] 10.222.0.105 () {56 vars in 1098 bytes} [Wed Aug 24 04:42:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839472d-7380-4696-a264-9d3075f941c0 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.262371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839472d-7380-4696-a264-9d3075f941c0 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.264496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a839472d-7380-4696-a264-9d3075f941c0 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.264496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 402/800] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:13 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:13.264496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839472d-7380-4696-a264-9d3075f941c0 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-196bf2d6-11f0-4bb0-a6d2-5531c0ce0f49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:13.275453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:13.276560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:13.277679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:13.277679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:13.277679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:13.277679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:13.277679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:13.277679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:13.277679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:13.277679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:13.281807 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 399/801] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:13.282629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bf2d6-11f0-4bb0-a6d2-5531c0ce0f49 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:13.337475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:13.380521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a839472d-7380-4696-a264-9d3075f941c0 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:13.387335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bf2d6-11f0-4bb0-a6d2-5531c0ce0f49 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:13.416212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8beda16-22f1-46a1-8b03-f1ca9b77f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:13.416838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beda16-22f1-46a1-8b03-f1ca9b77f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.417111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beda16-22f1-46a1-8b03-f1ca9b77f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.417379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8beda16-22f1-46a1-8b03-f1ca9b77f4c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.424423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a08db-a68c-438f-a291-7405f54278ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:13.425364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a08db-a68c-438f-a291-7405f54278ec None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.426069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a08db-a68c-438f-a291-7405f54278ec None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.426515 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a08db-a68c-438f-a291-7405f54278ec None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.427410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a08db-a68c-438f-a291-7405f54278ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:13.427860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a08db-a68c-438f-a291-7405f54278ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:13.428515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8beda16-22f1-46a1-8b03-f1ca9b77f4c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:13.429601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a08db-a68c-438f-a291-7405f54278ec None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b8beda16-22f1-46a1-8b03-f1ca9b77f4c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:13.433627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:13.434754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:42:13.436515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:13.440254 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 403/802] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:42:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:13.441577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9421a-b6ba-4851-944f-538fd12d8d7c None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.441798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9421a-b6ba-4851-944f-538fd12d8d7c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.442063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9421a-b6ba-4851-944f-538fd12d8d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.453451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 400/803] 10.222.0.105 () {56 vars in 1138 bytes} [Wed Aug 24 04:42:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:13.454468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a08db-a68c-438f-a291-7405f54278ec None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:13.464123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c0e9421a-b6ba-4851-944f-538fd12d8d7c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:13.465294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c0e9421a-b6ba-4851-944f-538fd12d8d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:13.507173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:13.547044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 404/804] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:13 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:13.547044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9421a-b6ba-4851-944f-538fd12d8d7c None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:13.588031 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a08db-a68c-438f-a291-7405f54278ec None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:13.605217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:13.636422 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ef345-5ba9-4a0c-b5e3-60bac1281fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:13.637071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ef345-5ba9-4a0c-b5e3-60bac1281fe3 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.637349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ef345-5ba9-4a0c-b5e3-60bac1281fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.637620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ef345-5ba9-4a0c-b5e3-60bac1281fe3 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.638830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ef345-5ba9-4a0c-b5e3-60bac1281fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:13.639161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ef345-5ba9-4a0c-b5e3-60bac1281fe3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:13.640732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ef345-5ba9-4a0c-b5e3-60bac1281fe3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:13.652643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 401/805] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:13.653220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ef345-5ba9-4a0c-b5e3-60bac1281fe3 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:13.673180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9421a-b6ba-4851-944f-538fd12d8d7c None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:13.717787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f2430-05cc-4828-96a7-76b832486c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:13.718320 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f2430-05cc-4828-96a7-76b832486c2e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.718641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f2430-05cc-4828-96a7-76b832486c2e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.718864 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f2430-05cc-4828-96a7-76b832486c2e None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.719562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f2430-05cc-4828-96a7-76b832486c2e None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:13.719934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f2430-05cc-4828-96a7-76b832486c2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:13.721215 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98f2430-05cc-4828-96a7-76b832486c2e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:13.730296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 405/806] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:42:13 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:13.731285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96945bbc-11b5-43fa-9d30-697f79cce701 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.731563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96945bbc-11b5-43fa-9d30-697f79cce701 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.733163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96945bbc-11b5-43fa-9d30-697f79cce701 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.738962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 406/807] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:13 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:13.738962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96945bbc-11b5-43fa-9d30-697f79cce701 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:13.770852 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ef345-5ba9-4a0c-b5e3-60bac1281fe3 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:13.773257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:13.804503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b0bfe-4d10-4984-8cef-93b5bf41fa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:13.805267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b0bfe-4d10-4984-8cef-93b5bf41fa66 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.805846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b0bfe-4d10-4984-8cef-93b5bf41fa66 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.806182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b0bfe-4d10-4984-8cef-93b5bf41fa66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.843380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96945bbc-11b5-43fa-9d30-697f79cce701 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:13.856034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:13.873073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b0bfe-4d10-4984-8cef-93b5bf41fa66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:13.874529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b0bfe-4d10-4984-8cef-93b5bf41fa66 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:13.876903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170696-c862-4c7c-afa1-766b806e8e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:13.877552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170696-c862-4c7c-afa1-766b806e8e24 None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.877823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170696-c862-4c7c-afa1-766b806e8e24 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.878089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15170696-c862-4c7c-afa1-766b806e8e24 None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/e6a894e4e6f24febbd8397fb8c4c7fb4/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.880194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15170696-c862-4c7c-afa1-766b806e8e24 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=e6a894e4e6f24febbd8397fb8c4c7fb4, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:13.881597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15170696-c862-4c7c-afa1-766b806e8e24 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:13.935318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 407/808] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:42:13 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/e6a894e4e6f24febbd8397fb8c4c7fb4/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:42:13.936353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170696-c862-4c7c-afa1-766b806e8e24 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:13.984548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 402/809] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:13 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:42:13.984548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96156b-fcaa-433d-af9c-a23fa9d010de None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:13.984548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96156b-fcaa-433d-af9c-a23fa9d010de None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:13.984548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee96156b-fcaa-433d-af9c-a23fa9d010de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:13.990215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:14.003541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ee96156b-fcaa-433d-af9c-a23fa9d010de None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:14.004753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ee96156b-fcaa-433d-af9c-a23fa9d010de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:14.058303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15170696-c862-4c7c-afa1-766b806e8e24 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.086270 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 403/810] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:13 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:14.087429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96156b-fcaa-433d-af9c-a23fa9d010de None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.101041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d8853-ed1d-4730-b281-cac865f2270b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.101758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d8853-ed1d-4730-b281-cac865f2270b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.102037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d8853-ed1d-4730-b281-cac865f2270b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.102305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d8853-ed1d-4730-b281-cac865f2270b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.115582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d8853-ed1d-4730-b281-cac865f2270b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.117025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d8853-ed1d-4730-b281-cac865f2270b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-771d8853-ed1d-4730-b281-cac865f2270b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:14.132073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:14.133168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:14.134285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:14.134285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:14.134285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:14.134285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 408/811] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:14.134554 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d8853-ed1d-4730-b281-cac865f2270b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.162195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:14.218667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee96156b-fcaa-433d-af9c-a23fa9d010de None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.255883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075cf45-68c8-4e95-a2a4-54edb1c215ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.256595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075cf45-68c8-4e95-a2a4-54edb1c215ff None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.256974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075cf45-68c8-4e95-a2a4-54edb1c215ff None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.257097 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d8853-ed1d-4730-b281-cac865f2270b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.257371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075cf45-68c8-4e95-a2a4-54edb1c215ff None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.258018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075cf45-68c8-4e95-a2a4-54edb1c215ff None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.260485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075cf45-68c8-4e95-a2a4-54edb1c215ff None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.262425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-3075cf45-68c8-4e95-a2a4-54edb1c215ff None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:14.271757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:14.275045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:14.275909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 404/812] 10.222.0.105 () {56 vars in 1134 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:14.276711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075cf45-68c8-4e95-a2a4-54edb1c215ff None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.301825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21743930-cda7-4426-87d1-6d83dbd44811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.302808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21743930-cda7-4426-87d1-6d83dbd44811 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.303290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21743930-cda7-4426-87d1-6d83dbd44811 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.303730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21743930-cda7-4426-87d1-6d83dbd44811 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.315679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21743930-cda7-4426-87d1-6d83dbd44811 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.317372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21743930-cda7-4426-87d1-6d83dbd44811 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-21743930-cda7-4426-87d1-6d83dbd44811 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:14.334457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:14.335477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:14.336363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:14.336363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:14.336363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:14.338558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 409/813] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:14.339608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21743930-cda7-4426-87d1-6d83dbd44811 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.383098 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075cf45-68c8-4e95-a2a4-54edb1c215ff None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.420521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05680b-1c7b-4b0c-9341-4aba1dc8bd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.421840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05680b-1c7b-4b0c-9341-4aba1dc8bd4e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.422338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05680b-1c7b-4b0c-9341-4aba1dc8bd4e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.422788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05680b-1c7b-4b0c-9341-4aba1dc8bd4e None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.423387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:14.424257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05680b-1c7b-4b0c-9341-4aba1dc8bd4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:14.424372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05680b-1c7b-4b0c-9341-4aba1dc8bd4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.425962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05680b-1c7b-4b0c-9341-4aba1dc8bd4e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.440802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 405/814] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:14.441615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05680b-1c7b-4b0c-9341-4aba1dc8bd4e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.451639 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21743930-cda7-4426-87d1-6d83dbd44811 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.488549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da85a8-758c-4063-8f1b-ac0f8954f76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.489156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da85a8-758c-4063-8f1b-ac0f8954f76d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.489436 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da85a8-758c-4063-8f1b-ac0f8954f76d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.489705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da85a8-758c-4063-8f1b-ac0f8954f76d None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.490335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da85a8-758c-4063-8f1b-ac0f8954f76d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:14.490634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da85a8-758c-4063-8f1b-ac0f8954f76d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.492108 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da85a8-758c-4063-8f1b-ac0f8954f76d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.501730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 410/815] 10.222.0.105 () {56 vars in 1129 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:14.502572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da85a8-758c-4063-8f1b-ac0f8954f76d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.508243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:14.556384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05680b-1c7b-4b0c-9341-4aba1dc8bd4e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.595682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187de93b-903c-4333-bc06-a29f842703c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.596343 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187de93b-903c-4333-bc06-a29f842703c1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.596630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187de93b-903c-4333-bc06-a29f842703c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.596874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187de93b-903c-4333-bc06-a29f842703c1 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.597502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187de93b-903c-4333-bc06-a29f842703c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:14.597786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187de93b-903c-4333-bc06-a29f842703c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.599151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187de93b-903c-4333-bc06-a29f842703c1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.604593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da85a8-758c-4063-8f1b-ac0f8954f76d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.614886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 406/816] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:14.614886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187de93b-903c-4333-bc06-a29f842703c1 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.644258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0e510-cf3c-40cc-93cc-35c4cdc3c33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.644977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0e510-cf3c-40cc-93cc-35c4cdc3c33d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.645326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0e510-cf3c-40cc-93cc-35c4cdc3c33d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.645666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0e510-cf3c-40cc-93cc-35c4cdc3c33d None admin] PATH_INFO: `/v3/services/network` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.646316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0e510-cf3c-40cc-93cc-35c4cdc3c33d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.647789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0e510-cf3c-40cc-93cc-35c4cdc3c33d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-efb0e510-cf3c-40cc-93cc-35c4cdc3c33d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:14.656496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:14.657665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 04:42:14.658724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:14.660181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 411/817] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:14.661020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0e510-cf3c-40cc-93cc-35c4cdc3c33d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.665461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:14.725626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187de93b-903c-4333-bc06-a29f842703c1 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.742827 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:14.761917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78304457-0bbd-4173-8722-e1debab3b2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.762545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0e510-cf3c-40cc-93cc-35c4cdc3c33d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.763357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78304457-0bbd-4173-8722-e1debab3b2d2 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.763610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78304457-0bbd-4173-8722-e1debab3b2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.763941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78304457-0bbd-4173-8722-e1debab3b2d2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.771848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78304457-0bbd-4173-8722-e1debab3b2d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.773749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78304457-0bbd-4173-8722-e1debab3b2d2 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-78304457-0bbd-4173-8722-e1debab3b2d2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:14.785936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:14.787089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 407/818] 10.222.0.105 () {56 vars in 1122 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:14.788058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78304457-0bbd-4173-8722-e1debab3b2d2 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.805168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8a03d-2ee9-46b6-993d-63131d687740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.805852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8a03d-2ee9-46b6-993d-63131d687740 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.806190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8a03d-2ee9-46b6-993d-63131d687740 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.806523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8a03d-2ee9-46b6-993d-63131d687740 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.807289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8a03d-2ee9-46b6-993d-63131d687740 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:14.807601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8a03d-2ee9-46b6-993d-63131d687740 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.809014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8a03d-2ee9-46b6-993d-63131d687740 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.819068 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 412/819] 10.222.0.105 () {56 vars in 1129 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:14.819068 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8a03d-2ee9-46b6-993d-63131d687740 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.894508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:14.905714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78304457-0bbd-4173-8722-e1debab3b2d2 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.928704 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8a03d-2ee9-46b6-993d-63131d687740 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:14.943748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba5a8b-dc09-4f00-8548-431da60ba330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.944461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba5a8b-dc09-4f00-8548-431da60ba330 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.944739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba5a8b-dc09-4f00-8548-431da60ba330 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.945008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba5a8b-dc09-4f00-8548-431da60ba330 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.952583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba5a8b-dc09-4f00-8548-431da60ba330 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.954316 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba5a8b-dc09-4f00-8548-431da60ba330 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.963933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df256dc-4bea-42a6-a267-3b05393ba586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:14.964611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df256dc-4bea-42a6-a267-3b05393ba586 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:14.964955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df256dc-4bea-42a6-a267-3b05393ba586 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:14.965292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df256dc-4bea-42a6-a267-3b05393ba586 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-42ba5a8b-dc09-4f00-8548-431da60ba330 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:14.966080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:14.967121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:14.968043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:14.968043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:14.968043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:14.968043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:14.968043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:14.968043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:14.968043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:14.968043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:14.968472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df256dc-4bea-42a6-a267-3b05393ba586 None admin] RBAC: Adding query filter params (name=network) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:14.968472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df256dc-4bea-42a6-a267-3b05393ba586 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:14.968595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 408/820] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:14.968595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba5a8b-dc09-4f00-8548-431da60ba330 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:14.973113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df256dc-4bea-42a6-a267-3b05393ba586 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:14.982945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 413/821] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/services?name=network => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:14.982945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df256dc-4bea-42a6-a267-3b05393ba586 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:15.017522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:15.085114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba5a8b-dc09-4f00-8548-431da60ba330 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:15.089731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df256dc-4bea-42a6-a267-3b05393ba586 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:15.121245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:15.125813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e3df3-02e0-448a-b621-2b22f7345626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:15.126527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e3df3-02e0-448a-b621-2b22f7345626 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:15.126789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f89fb-8f06-409e-96a9-1eb95dc62381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:15.127012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e3df3-02e0-448a-b621-2b22f7345626 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:15.127404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e3df3-02e0-448a-b621-2b22f7345626 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:15.127511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f89fb-8f06-409e-96a9-1eb95dc62381 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:15.127689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f89fb-8f06-409e-96a9-1eb95dc62381 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:15.127955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f89fb-8f06-409e-96a9-1eb95dc62381 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:15.128519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e3df3-02e0-448a-b621-2b22f7345626 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:15.128614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f89fb-8f06-409e-96a9-1eb95dc62381 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:15.128986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e3df3-02e0-448a-b621-2b22f7345626 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:15.129929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f89fb-8f06-409e-96a9-1eb95dc62381 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:15.130669 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e3df3-02e0-448a-b621-2b22f7345626 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:15.148760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 409/822] 10.222.0.105 () {56 vars in 1154 bytes} [Wed Aug 24 04:42:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:15.149578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e3df3-02e0-448a-b621-2b22f7345626 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:15.237839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 414/823] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:42:14 2022] POST /identity/v3/endpoints => generated 348 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:15.239080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f89fb-8f06-409e-96a9-1eb95dc62381 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:15.268274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:15.275803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e3df3-02e0-448a-b621-2b22f7345626 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:15.339087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1020825-ce20-4b9b-ba1c-96a280bffada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:15.340549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1020825-ce20-4b9b-ba1c-96a280bffada None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:15.341168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1020825-ce20-4b9b-ba1c-96a280bffada None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:15.342329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1020825-ce20-4b9b-ba1c-96a280bffada None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:15.343610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1020825-ce20-4b9b-ba1c-96a280bffada None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:15.344194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1020825-ce20-4b9b-ba1c-96a280bffada None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:15.347129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1020825-ce20-4b9b-ba1c-96a280bffada None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:15.364822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 410/824] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:15.366107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1020825-ce20-4b9b-ba1c-96a280bffada None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:15.391388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f89fb-8f06-409e-96a9-1eb95dc62381 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:15.436458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6f5c8-4acb-4f66-abce-b1d8964a5637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:15.436458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6f5c8-4acb-4f66-abce-b1d8964a5637 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:15.436758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6f5c8-4acb-4f66-abce-b1d8964a5637 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:15.437003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6f5c8-4acb-4f66-abce-b1d8964a5637 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:15.437487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6f5c8-4acb-4f66-abce-b1d8964a5637 None admin] RBAC: Adding query filter params (type=network) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:15.437783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6f5c8-4acb-4f66-abce-b1d8964a5637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:15.439177 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6f5c8-4acb-4f66-abce-b1d8964a5637 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:15.451113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 415/825] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:15 2022] GET /identity/v3/services?type=network => generated 352 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:15.451113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6f5c8-4acb-4f66-abce-b1d8964a5637 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:15.499574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1020825-ce20-4b9b-ba1c-96a280bffada None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:15.544490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c91988-0e18-4ef3-aa82-24e5a0d5b3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:15.544991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c91988-0e18-4ef3-aa82-24e5a0d5b3d1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:15.545268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c91988-0e18-4ef3-aa82-24e5a0d5b3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:15.545534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c91988-0e18-4ef3-aa82-24e5a0d5b3d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:15.559308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c91988-0e18-4ef3-aa82-24e5a0d5b3d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:15.561251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c91988-0e18-4ef3-aa82-24e5a0d5b3d1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:15.579642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6f5c8-4acb-4f66-abce-b1d8964a5637 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-c2c91988-0e18-4ef3-aa82-24e5a0d5b3d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:15.580699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:15.581847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:15.587606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:15.587606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:15.587606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:15.587606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 411/826] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:15.587606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c91988-0e18-4ef3-aa82-24e5a0d5b3d1 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:15.588480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:15.592762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:15.634842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534e36c-3a0e-4435-8591-0bdcd42ca246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:15.635757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e36c-3a0e-4435-8591-0bdcd42ca246 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:15.636205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e36c-3a0e-4435-8591-0bdcd42ca246 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:15.636628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534e36c-3a0e-4435-8591-0bdcd42ca246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:15.638064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534e36c-3a0e-4435-8591-0bdcd42ca246 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=3c822c8f23b74bae84769046fdb85db4) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:15.638521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534e36c-3a0e-4435-8591-0bdcd42ca246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:15.641699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534e36c-3a0e-4435-8591-0bdcd42ca246 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:15.652716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 416/827] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:15 2022] GET /identity/v3/role_assignments?user.id=3c822c8f23b74bae84769046fdb85db4&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 610 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:15.654434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534e36c-3a0e-4435-8591-0bdcd42ca246 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:15.760802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c91988-0e18-4ef3-aa82-24e5a0d5b3d1 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:15.817154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0b636-a2f1-4fc7-a695-680d3a6f3818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:15.818827 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0b636-a2f1-4fc7-a695-680d3a6f3818 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:15.819129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0b636-a2f1-4fc7-a695-680d3a6f3818 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:15.819404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0b636-a2f1-4fc7-a695-680d3a6f3818 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:15.820079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0b636-a2f1-4fc7-a695-680d3a6f3818 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:15.821456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0b636-a2f1-4fc7-a695-680d3a6f3818 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:15.848394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534e36c-3a0e-4435-8591-0bdcd42ca246 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:15.869020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:15.890535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f370ab-05fa-4533-b984-eb4049f9342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:15.891271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f370ab-05fa-4533-b984-eb4049f9342c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:15.891665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f370ab-05fa-4533-b984-eb4049f9342c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:15.892075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f370ab-05fa-4533-b984-eb4049f9342c None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:15.892780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f370ab-05fa-4533-b984-eb4049f9342c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:15.893240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f370ab-05fa-4533-b984-eb4049f9342c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:15.894774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f370ab-05fa-4533-b984-eb4049f9342c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:15.908617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 417/828] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:15.914324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f370ab-05fa-4533-b984-eb4049f9342c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:15.933554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 412/829] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:42:15 2022] POST /identity/v3/endpoints => generated 339 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:15.940414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce5155-c9b4-457b-b037-2193d9ce41cf None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:15.940703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce5155-c9b4-457b-b037-2193d9ce41cf None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:15.940974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce5155-c9b4-457b-b037-2193d9ce41cf None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:15.942934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 413/830] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:15.949133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407d13-3dca-426b-a4d7-9e5e50dcbcd8 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:15.949133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407d13-3dca-426b-a4d7-9e5e50dcbcd8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:15.949425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407d13-3dca-426b-a4d7-9e5e50dcbcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:15.971132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b5407d13-3dca-426b-a4d7-9e5e50dcbcd8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:15.972459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b5407d13-3dca-426b-a4d7-9e5e50dcbcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:16.051351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f370ab-05fa-4533-b984-eb4049f9342c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:16.052398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:16.101429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 414/831] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:15 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:16.118764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc196675-3943-42cb-bb70-d3ccf7b44013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:16.119451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc196675-3943-42cb-bb70-d3ccf7b44013 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:16.119729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc196675-3943-42cb-bb70-d3ccf7b44013 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:16.120001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc196675-3943-42cb-bb70-d3ccf7b44013 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:16.128126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc196675-3943-42cb-bb70-d3ccf7b44013 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:16.129572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc196675-3943-42cb-bb70-d3ccf7b44013 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-fc196675-3943-42cb-bb70-d3ccf7b44013 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:16.145144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:16.146337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:16.147462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:16.147462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:16.147462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:16.147462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:16.147462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:16.147462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:16.147462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:16.147462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:16.147462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 418/832] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:16.152326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407d13-3dca-426b-a4d7-9e5e50dcbcd8 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:16.262376 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407d13-3dca-426b-a4d7-9e5e50dcbcd8 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:16.304686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbace48-74b1-48b2-8e31-fa70e4a780c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:16.305965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbace48-74b1-48b2-8e31-fa70e4a780c0 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:16.306417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbace48-74b1-48b2-8e31-fa70e4a780c0 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:16.306831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbace48-74b1-48b2-8e31-fa70e4a780c0 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:16.307691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbace48-74b1-48b2-8e31-fa70e4a780c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:16.308136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbace48-74b1-48b2-8e31-fa70e4a780c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:16.310319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbace48-74b1-48b2-8e31-fa70e4a780c0 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:16.323972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 415/833] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:16.330076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc196675-3943-42cb-bb70-d3ccf7b44013 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:16.377511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:16.442641 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbace48-74b1-48b2-8e31-fa70e4a780c0 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:16.461175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc196675-3943-42cb-bb70-d3ccf7b44013 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:16.506704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:16.509781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cd98d-887b-4818-87dd-803b14bc192d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:16.510563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cd98d-887b-4818-87dd-803b14bc192d None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:16.510937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cd98d-887b-4818-87dd-803b14bc192d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:16.511340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948cd98d-887b-4818-87dd-803b14bc192d None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/da66e763177c47e39b3475fc42ccc8f0/roles/4f58cf3fe09f44b1b59e224e7077ce04` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:16.530495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cd98d-887b-4818-87dd-803b14bc192d None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=da66e763177c47e39b3475fc42ccc8f0, role_id=4f58cf3fe09f44b1b59e224e7077ce04)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:16.532823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948cd98d-887b-4818-87dd-803b14bc192d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:16.568994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbace48-74b1-48b2-8e31-fa70e4a780c0 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:16.603228 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ff387-89b4-4f39-b39c-a637b3862f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:16.603891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff387-89b4-4f39-b39c-a637b3862f31 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:16.604125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff387-89b4-4f39-b39c-a637b3862f31 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:16.604448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ff387-89b4-4f39-b39c-a637b3862f31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:16.613567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 419/834] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:42:16 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/da66e763177c47e39b3475fc42ccc8f0/roles/4f58cf3fe09f44b1b59e224e7077ce04 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:42:16.620388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ff387-89b4-4f39-b39c-a637b3862f31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-095ff387-89b4-4f39-b39c-a637b3862f31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:16.626774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:16.628076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:16.629218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 416/835] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:16.635389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cd98d-887b-4818-87dd-803b14bc192d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:16.734363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:16.747353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948cd98d-887b-4818-87dd-803b14bc192d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:16.783564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ad75a-99c9-46a8-8162-998e362fdcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:16.784177 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ad75a-99c9-46a8-8162-998e362fdcbc None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:16.784456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ad75a-99c9-46a8-8162-998e362fdcbc None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:16.784729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ad75a-99c9-46a8-8162-998e362fdcbc None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:16.785397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ad75a-99c9-46a8-8162-998e362fdcbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:16.785698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ad75a-99c9-46a8-8162-998e362fdcbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:16.787147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ad75a-99c9-46a8-8162-998e362fdcbc None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:16.796459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 420/836] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:42:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:16.802058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ff387-89b4-4f39-b39c-a637b3862f31 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:16.912319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ff387-89b4-4f39-b39c-a637b3862f31 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:16.920254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:16.952071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e8732-91a6-4659-8021-c9c3368ddaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:16.952701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e8732-91a6-4659-8021-c9c3368ddaea None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:16.952976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e8732-91a6-4659-8021-c9c3368ddaea None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:16.953242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169e8732-91a6-4659-8021-c9c3368ddaea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:16.963776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e8732-91a6-4659-8021-c9c3368ddaea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:16.965241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169e8732-91a6-4659-8021-c9c3368ddaea None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-169e8732-91a6-4659-8021-c9c3368ddaea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:16.982716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:16.983766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:16.985272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:16.985272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:16.985272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:16.985272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 417/837] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:16.989962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ad75a-99c9-46a8-8162-998e362fdcbc None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:17.098613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ad75a-99c9-46a8-8162-998e362fdcbc None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:17.134293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929d488-b608-4810-a8f0-3f1c21410de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:17.134912 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929d488-b608-4810-a8f0-3f1c21410de4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:17.135224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929d488-b608-4810-a8f0-3f1c21410de4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:17.135809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929d488-b608-4810-a8f0-3f1c21410de4 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:17.136607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929d488-b608-4810-a8f0-3f1c21410de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:17.137063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929d488-b608-4810-a8f0-3f1c21410de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:17.138760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929d488-b608-4810-a8f0-3f1c21410de4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:17.151487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 421/838] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:17.157296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e8732-91a6-4659-8021-c9c3368ddaea None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:17.265001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169e8732-91a6-4659-8021-c9c3368ddaea None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:17.301026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943f2e9-9ccb-477c-b20b-a89ce778ec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:17.302041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943f2e9-9ccb-477c-b20b-a89ce778ec8f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:17.302609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943f2e9-9ccb-477c-b20b-a89ce778ec8f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:17.303114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943f2e9-9ccb-477c-b20b-a89ce778ec8f None admin] PATH_INFO: `/v3/users/swift` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:17.312282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943f2e9-9ccb-477c-b20b-a89ce778ec8f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:17.314444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b943f2e9-9ccb-477c-b20b-a89ce778ec8f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:17.321312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65800b-f34c-4c49-b4a1-008503b6c7f8 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:17.321884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65800b-f34c-4c49-b4a1-008503b6c7f8 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:17.322362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65800b-f34c-4c49-b4a1-008503b6c7f8 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:17.324445 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 422/839] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:17 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-b943f2e9-9ccb-477c-b20b-a89ce778ec8f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:17.329993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:17.330972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:17.331963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 418/840] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:17.334324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df616b8b-8351-46df-8d80-74e853323f38 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:17.334754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df616b8b-8351-46df-8d80-74e853323f38 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:17.335198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df616b8b-8351-46df-8d80-74e853323f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:17.337084 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943f2e9-9ccb-477c-b20b-a89ce778ec8f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:17.353778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-df616b8b-8351-46df-8d80-74e853323f38 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:17.355428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-df616b8b-8351-46df-8d80-74e853323f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:17.359324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:17.360776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:17.427379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 423/841] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:17 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:17.468033 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943f2e9-9ccb-477c-b20b-a89ce778ec8f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:17.503543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ceb577-5053-4d08-8969-c3ed95df457a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:17.504676 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ceb577-5053-4d08-8969-c3ed95df457a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:17.505163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ceb577-5053-4d08-8969-c3ed95df457a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:17.505631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ceb577-5053-4d08-8969-c3ed95df457a None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:17.506597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ceb577-5053-4d08-8969-c3ed95df457a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:17.507133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ceb577-5053-4d08-8969-c3ed95df457a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:17.508823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ceb577-5053-4d08-8969-c3ed95df457a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:17.533464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 419/842] 10.222.0.105 () {56 vars in 1140 bytes} [Wed Aug 24 04:42:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:17.539231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df616b8b-8351-46df-8d80-74e853323f38 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:17.644211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df616b8b-8351-46df-8d80-74e853323f38 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:17.681392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc868d5-6d1c-44c7-9d82-ce397c1450d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:17.682015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc868d5-6d1c-44c7-9d82-ce397c1450d7 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:17.682295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc868d5-6d1c-44c7-9d82-ce397c1450d7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:17.682566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc868d5-6d1c-44c7-9d82-ce397c1450d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:17.691375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:17.748833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ceb577-5053-4d08-8969-c3ed95df457a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:17.757107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc868d5-6d1c-44c7-9d82-ce397c1450d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:17.759156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc868d5-6d1c-44c7-9d82-ce397c1450d7 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:17.835279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:17.864840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ceb577-5053-4d08-8969-c3ed95df457a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:17.887507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 424/843] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:17 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:42:17.893220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc868d5-6d1c-44c7-9d82-ce397c1450d7 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:17.911352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794dab57-4f77-4632-a2e7-962fccaf6966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:17.911986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dab57-4f77-4632-a2e7-962fccaf6966 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:17.912280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dab57-4f77-4632-a2e7-962fccaf6966 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:17.912527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794dab57-4f77-4632-a2e7-962fccaf6966 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:17.913188 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794dab57-4f77-4632-a2e7-962fccaf6966 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:17.914453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794dab57-4f77-4632-a2e7-962fccaf6966 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-794dab57-4f77-4632-a2e7-962fccaf6966 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:17.923930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:17.924902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:17.925744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 420/844] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:42:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:17.926554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d4e95-f7bc-4e42-8901-260e6663a179 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:17.926554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d4e95-f7bc-4e42-8901-260e6663a179 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:17.926685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d4e95-f7bc-4e42-8901-260e6663a179 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:17.927974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 421/845] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:17 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:17.932567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d4e95-f7bc-4e42-8901-260e6663a179 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:17.942758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:18.008083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc868d5-6d1c-44c7-9d82-ce397c1450d7 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:18.043986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d4e95-f7bc-4e42-8901-260e6663a179 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:18.044196 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c526e-31d8-4c97-a242-034e1df114fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:18.044782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c526e-31d8-4c97-a242-034e1df114fc None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.045061 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c526e-31d8-4c97-a242-034e1df114fc None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.045407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c526e-31d8-4c97-a242-034e1df114fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.055912 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c526e-31d8-4c97-a242-034e1df114fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:18.057344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c526e-31d8-4c97-a242-034e1df114fc None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-5a4c526e-31d8-4c97-a242-034e1df114fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:18.074576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:18.075641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:18.076563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:18.076563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:18.076563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:18.076563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 425/846] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:18.077221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964a424-02a5-4e3f-9f44-100432630f50 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.077397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964a424-02a5-4e3f-9f44-100432630f50 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.077846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964a424-02a5-4e3f-9f44-100432630f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.085929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1edeb-469b-479b-a43d-0706b1a6b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:18.086686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1edeb-469b-479b-a43d-0706b1a6b422 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.087104 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1edeb-469b-479b-a43d-0706b1a6b422 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.087470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1edeb-469b-479b-a43d-0706b1a6b422 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.088222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1edeb-469b-479b-a43d-0706b1a6b422 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:18.088615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1edeb-469b-479b-a43d-0706b1a6b422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:18.090544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1edeb-469b-479b-a43d-0706b1a6b422 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:18.095260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-0964a424-02a5-4e3f-9f44-100432630f50 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:18.095679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 422/847] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:42:17 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:18.096449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1edeb-469b-479b-a43d-0706b1a6b422 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:18.100922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:18.101044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0964a424-02a5-4e3f-9f44-100432630f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:18.179180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 426/848] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:18 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:18.180080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964a424-02a5-4e3f-9f44-100432630f50 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:18.208287 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:18.219603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1edeb-469b-479b-a43d-0706b1a6b422 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:18.257477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e5e67-688b-4c59-8a65-6f439694461e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:18.258294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e5e67-688b-4c59-8a65-6f439694461e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.258616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e5e67-688b-4c59-8a65-6f439694461e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.259127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e5e67-688b-4c59-8a65-6f439694461e None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.260065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e5e67-688b-4c59-8a65-6f439694461e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:18.260459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e5e67-688b-4c59-8a65-6f439694461e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:18.262146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e5e67-688b-4c59-8a65-6f439694461e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:18.280171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 423/849] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:18.282345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e5e67-688b-4c59-8a65-6f439694461e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:18.308608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964a424-02a5-4e3f-9f44-100432630f50 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:18.346566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8e66a-ed56-4ac9-91c7-0f8f1d6e7d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:18.347412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8e66a-ed56-4ac9-91c7-0f8f1d6e7d65 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.347550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8e66a-ed56-4ac9-91c7-0f8f1d6e7d65 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.347816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a8e66a-ed56-4ac9-91c7-0f8f1d6e7d65 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.348500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8e66a-ed56-4ac9-91c7-0f8f1d6e7d65 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:18.348805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8e66a-ed56-4ac9-91c7-0f8f1d6e7d65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:18.350171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a8e66a-ed56-4ac9-91c7-0f8f1d6e7d65 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:18.354886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 427/850] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:42:18 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:18.362054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:18.409093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e5e67-688b-4c59-8a65-6f439694461e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:18.448615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4bd27-6f33-4dca-83bb-d6ae66b517c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:18.449644 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4bd27-6f33-4dca-83bb-d6ae66b517c5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.450169 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4bd27-6f33-4dca-83bb-d6ae66b517c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.450638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4bd27-6f33-4dca-83bb-d6ae66b517c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.452119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8e66a-ed56-4ac9-91c7-0f8f1d6e7d65 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:18.459693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4bd27-6f33-4dca-83bb-d6ae66b517c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:18.461158 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4bd27-6f33-4dca-83bb-d6ae66b517c5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-53e4bd27-6f33-4dca-83bb-d6ae66b517c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:18.477242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:18.478213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:18.479207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:18.479207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:18.479207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:18.479207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:18.479207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:18.479207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:18.479207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:18.479207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:18.481956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 424/851] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:18.487956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4bd27-6f33-4dca-83bb-d6ae66b517c5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:18.557744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:18.575677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a8e66a-ed56-4ac9-91c7-0f8f1d6e7d65 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:18.614604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78e694d-8cd8-436a-a804-d2ee97d07b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:18.615265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e694d-8cd8-436a-a804-d2ee97d07b0f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.615538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e694d-8cd8-436a-a804-d2ee97d07b0f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.615801 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78e694d-8cd8-436a-a804-d2ee97d07b0f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.616521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e694d-8cd8-436a-a804-d2ee97d07b0f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:18.618473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78e694d-8cd8-436a-a804-d2ee97d07b0f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:18.619933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4bd27-6f33-4dca-83bb-d6ae66b517c5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-a78e694d-8cd8-436a-a804-d2ee97d07b0f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:18.635972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:18.637989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:18.638998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 428/852] 10.222.0.105 () {56 vars in 1137 bytes} [Wed Aug 24 04:42:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:18.644873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ff1d7-214e-491c-b139-bbdc341a1819 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.644873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ff1d7-214e-491c-b139-bbdc341a1819 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.644873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ff1d7-214e-491c-b139-bbdc341a1819 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.646165 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 429/853] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:18.646948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ff1d7-214e-491c-b139-bbdc341a1819 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:18.650735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:18.662748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a104c3-81c2-49cc-8e53-2108d7698044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:18.663390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a104c3-81c2-49cc-8e53-2108d7698044 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.663684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a104c3-81c2-49cc-8e53-2108d7698044 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.663931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a104c3-81c2-49cc-8e53-2108d7698044 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.664694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a104c3-81c2-49cc-8e53-2108d7698044 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:18.664900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a104c3-81c2-49cc-8e53-2108d7698044 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:18.666312 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a104c3-81c2-49cc-8e53-2108d7698044 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:18.678363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 425/854] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:18.679929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c650d6-9151-499c-bc6c-53109f2209c0 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.679929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c650d6-9151-499c-bc6c-53109f2209c0 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.680133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c650d6-9151-499c-bc6c-53109f2209c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.700251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-d4c650d6-9151-499c-bc6c-53109f2209c0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:18.701440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d4c650d6-9151-499c-bc6c-53109f2209c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:18.766448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ff1d7-214e-491c-b139-bbdc341a1819 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:18.774824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 426/855] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:18 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:18.775983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c650d6-9151-499c-bc6c-53109f2209c0 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:18.807383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013a761-0f8f-402e-99b5-7415ff6c77e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:18.808354 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013a761-0f8f-402e-99b5-7415ff6c77e3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.808945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013a761-0f8f-402e-99b5-7415ff6c77e3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.809402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013a761-0f8f-402e-99b5-7415ff6c77e3 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.810449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013a761-0f8f-402e-99b5-7415ff6c77e3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:18.811051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013a761-0f8f-402e-99b5-7415ff6c77e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:18.812848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013a761-0f8f-402e-99b5-7415ff6c77e3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:18.818566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 430/856] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:18 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:18.826942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013a761-0f8f-402e-99b5-7415ff6c77e3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:18.848931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:18.888172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c650d6-9151-499c-bc6c-53109f2209c0 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:18.905533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:18.927963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a5a4-f362-4686-b492-116ac0da34da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:18.928919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a5a4-f362-4686-b492-116ac0da34da None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.929615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a5a4-f362-4686-b492-116ac0da34da None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.930108 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a5a4-f362-4686-b492-116ac0da34da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:18.931051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337a5a4-f362-4686-b492-116ac0da34da None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=e6a894e4e6f24febbd8397fb8c4c7fb4) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:18.931675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337a5a4-f362-4686-b492-116ac0da34da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:18.934952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337a5a4-f362-4686-b492-116ac0da34da None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:18.942298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 427/857] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:18 2022] GET /identity/v3/role_assignments?user.id=e6a894e4e6f24febbd8397fb8c4c7fb4&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 610 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:18.957603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013a761-0f8f-402e-99b5-7415ff6c77e3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:18.998118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2dc0d-6848-46a6-acce-9196e29b5ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:18.998758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2dc0d-6848-46a6-acce-9196e29b5ac5 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:18.999058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2dc0d-6848-46a6-acce-9196e29b5ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:18.999430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2dc0d-6848-46a6-acce-9196e29b5ac5 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:19.000098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2dc0d-6848-46a6-acce-9196e29b5ac5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:19.000406 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2dc0d-6848-46a6-acce-9196e29b5ac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:19.001816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2dc0d-6848-46a6-acce-9196e29b5ac5 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:19.007515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 431/858] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:18 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:19.064638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a5a4-f362-4686-b492-116ac0da34da None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:19.181616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a5a4-f362-4686-b492-116ac0da34da None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:19.217026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf928a-eb91-42b2-b82f-7e3bd7049169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:19.217744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf928a-eb91-42b2-b82f-7e3bd7049169 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:19.218092 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf928a-eb91-42b2-b82f-7e3bd7049169 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:19.218434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf928a-eb91-42b2-b82f-7e3bd7049169 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:19.228793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf928a-eb91-42b2-b82f-7e3bd7049169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-ddaf928a-eb91-42b2-b82f-7e3bd7049169 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:19.234163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:19.235346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:42:19.236384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:19.237297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 428/859] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:42:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:19.242956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2dc0d-6848-46a6-acce-9196e29b5ac5 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:19.328350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:19.352305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2dc0d-6848-46a6-acce-9196e29b5ac5 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:19.389338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c8b54-5ccd-4ff9-b1f8-3484102b0aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:19.389952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c8b54-5ccd-4ff9-b1f8-3484102b0aa2 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:19.390292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c8b54-5ccd-4ff9-b1f8-3484102b0aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:19.390563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03c8b54-5ccd-4ff9-b1f8-3484102b0aa2 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:19.391322 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c8b54-5ccd-4ff9-b1f8-3484102b0aa2 None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:19.391631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c8b54-5ccd-4ff9-b1f8-3484102b0aa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:19.392994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03c8b54-5ccd-4ff9-b1f8-3484102b0aa2 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:19.402004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 432/860] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:42:19 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:19.407585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf928a-eb91-42b2-b82f-7e3bd7049169 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:19.448990 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:19.514915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf928a-eb91-42b2-b82f-7e3bd7049169 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:19.554820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3a2bc-2d07-4059-b5fc-ccd29f7d6f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:19.555481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a2bc-2d07-4059-b5fc-ccd29f7d6f3a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:19.555755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a2bc-2d07-4059-b5fc-ccd29f7d6f3a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:19.556022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3a2bc-2d07-4059-b5fc-ccd29f7d6f3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:19.567173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3a2bc-2d07-4059-b5fc-ccd29f7d6f3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:19.568605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3a2bc-2d07-4059-b5fc-ccd29f7d6f3a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-93e3a2bc-2d07-4059-b5fc-ccd29f7d6f3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:19.583307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:19.584237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:19.585461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:19.585461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:19.585461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:19.585461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 429/861] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:19.590685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c8b54-5ccd-4ff9-b1f8-3484102b0aa2 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:19.693326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03c8b54-5ccd-4ff9-b1f8-3484102b0aa2 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:19.728202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8448fb-7e27-42d2-b509-57058ed69bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:19.728811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8448fb-7e27-42d2-b509-57058ed69bb5 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:19.729092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8448fb-7e27-42d2-b509-57058ed69bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:19.729368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8448fb-7e27-42d2-b509-57058ed69bb5 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:19.730009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8448fb-7e27-42d2-b509-57058ed69bb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:19.730313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8448fb-7e27-42d2-b509-57058ed69bb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:19.731919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8448fb-7e27-42d2-b509-57058ed69bb5 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:19.739321 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:19.744456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 433/862] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:19.750871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3a2bc-2d07-4059-b5fc-ccd29f7d6f3a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:19.847645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:19.857354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3a2bc-2d07-4059-b5fc-ccd29f7d6f3a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:19.892510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2043998-287a-4ab2-8fc3-7c69a0752d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:19.893143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2043998-287a-4ab2-8fc3-7c69a0752d94 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:19.893418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2043998-287a-4ab2-8fc3-7c69a0752d94 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:19.893688 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2043998-287a-4ab2-8fc3-7c69a0752d94 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:19.900962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2043998-287a-4ab2-8fc3-7c69a0752d94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:19.902723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2043998-287a-4ab2-8fc3-7c69a0752d94 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-c2043998-287a-4ab2-8fc3-7c69a0752d94 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:19.914480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:19.915462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:19.916998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 430/863] 10.222.0.105 () {56 vars in 1122 bytes} [Wed Aug 24 04:42:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:19.921632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8448fb-7e27-42d2-b509-57058ed69bb5 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:20.024129 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8448fb-7e27-42d2-b509-57058ed69bb5 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:20.060170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635868f-43f7-49e0-86a2-20734818904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:20.060782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635868f-43f7-49e0-86a2-20734818904d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:20.061059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635868f-43f7-49e0-86a2-20734818904d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:20.061393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635868f-43f7-49e0-86a2-20734818904d None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:20.062137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635868f-43f7-49e0-86a2-20734818904d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:20.062436 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635868f-43f7-49e0-86a2-20734818904d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:20.063874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635868f-43f7-49e0-86a2-20734818904d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:20.086889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 434/864] 10.222.0.105 () {56 vars in 1154 bytes} [Wed Aug 24 04:42:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:20.092487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2043998-287a-4ab2-8fc3-7c69a0752d94 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:20.200537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2043998-287a-4ab2-8fc3-7c69a0752d94 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:20.237532 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaa7a8-e133-48f6-a431-998392d76be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:20.238141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaa7a8-e133-48f6-a431-998392d76be9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:20.238420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaa7a8-e133-48f6-a431-998392d76be9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:20.238690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acaa7a8-e133-48f6-a431-998392d76be9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:20.258445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:20.307507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaa7a8-e133-48f6-a431-998392d76be9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:20.309596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acaa7a8-e133-48f6-a431-998392d76be9 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:20.329071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bd707-2b58-4aa4-94ea-ee1fddd24230 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:20.329369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bd707-2b58-4aa4-94ea-ee1fddd24230 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:20.329623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0bd707-2b58-4aa4-94ea-ee1fddd24230 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:20.331403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 435/865] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:20.336504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35dc6d-4476-413d-b47a-436692786f3f None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:20.336793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35dc6d-4476-413d-b47a-436692786f3f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:20.337071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35dc6d-4476-413d-b47a-436692786f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:20.356440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-9e35dc6d-4476-413d-b47a-436692786f3f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:20.357706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9e35dc6d-4476-413d-b47a-436692786f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:20.413845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:20.422454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 431/866] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:20 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:42:20.423618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaa7a8-e133-48f6-a431-998392d76be9 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:20.447222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 436/867] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:20 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:20.492540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:20.533588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acaa7a8-e133-48f6-a431-998392d76be9 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:20.570220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f836d-fce5-4748-996f-37c4b1843310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:20.570835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f836d-fce5-4748-996f-37c4b1843310 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:20.571141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f836d-fce5-4748-996f-37c4b1843310 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:20.571414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f836d-fce5-4748-996f-37c4b1843310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:20.582394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f836d-fce5-4748-996f-37c4b1843310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:20.583911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f836d-fce5-4748-996f-37c4b1843310 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-f24f836d-fce5-4748-996f-37c4b1843310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:20.598798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:20.600600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:20.602329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:20.602329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:20.602329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:20.602329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 432/868] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:20.612986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35dc6d-4476-413d-b47a-436692786f3f None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:20.711533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f836d-fce5-4748-996f-37c4b1843310 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:20.800410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35dc6d-4476-413d-b47a-436692786f3f None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:20.818041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f836d-fce5-4748-996f-37c4b1843310 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:20.819797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:20.853694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f4c1a-de36-410d-95b0-c11fe92916b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:20.854389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f4c1a-de36-410d-95b0-c11fe92916b8 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:20.854725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f4c1a-de36-410d-95b0-c11fe92916b8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:20.855127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f4c1a-de36-410d-95b0-c11fe92916b8 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:20.856124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f4c1a-de36-410d-95b0-c11fe92916b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:20.857775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f4c1a-de36-410d-95b0-c11fe92916b8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:20.868497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97571e-b0de-4c3f-9bc8-d3b4b6546dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:20.869503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97571e-b0de-4c3f-9bc8-d3b4b6546dae None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:20.869982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97571e-b0de-4c3f-9bc8-d3b4b6546dae None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:20.870456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97571e-b0de-4c3f-9bc8-d3b4b6546dae None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:20.871804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97571e-b0de-4c3f-9bc8-d3b4b6546dae None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:20.872336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97571e-b0de-4c3f-9bc8-d3b4b6546dae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:20.874850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97571e-b0de-4c3f-9bc8-d3b4b6546dae None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:20.892803 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 437/869] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:20.898447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97571e-b0de-4c3f-9bc8-d3b4b6546dae None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:20.950074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:20.955451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 433/870] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:42:20 2022] POST /identity/v3/services => generated 252 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:42:21.021348 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97571e-b0de-4c3f-9bc8-d3b4b6546dae None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:21.075632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e527be6-018e-487f-b102-60c24793b9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:21.076303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e527be6-018e-487f-b102-60c24793b9bd None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:21.076647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e527be6-018e-487f-b102-60c24793b9bd None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:21.076922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e527be6-018e-487f-b102-60c24793b9bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:21.085116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e527be6-018e-487f-b102-60c24793b9bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:21.086601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e527be6-018e-487f-b102-60c24793b9bd None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-7e527be6-018e-487f-b102-60c24793b9bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:21.098737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:21.099703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:21.100587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:21.100587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:21.100587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:21.100587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:21.100587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:21.100587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:21.100587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:21.100587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:21.100587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 438/871] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:21.106563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f4c1a-de36-410d-95b0-c11fe92916b8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:21.137324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5418a-3476-4310-bb91-85224b674048 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:21.138051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5418a-3476-4310-bb91-85224b674048 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:21.138491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5418a-3476-4310-bb91-85224b674048 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:21.140287 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 439/872] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:21.145765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27668fb-3654-4de6-912b-f2c1cb462309 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:21.146291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27668fb-3654-4de6-912b-f2c1cb462309 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:21.146761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27668fb-3654-4de6-912b-f2c1cb462309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:21.170267 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-a27668fb-3654-4de6-912b-f2c1cb462309 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:21.171540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a27668fb-3654-4de6-912b-f2c1cb462309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:21.233672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f4c1a-de36-410d-95b0-c11fe92916b8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:21.261166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 440/873] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:21 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:21.263392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7f77c-4445-4aaf-b50d-b0f5fbb460dc None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:21.263952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7f77c-4445-4aaf-b50d-b0f5fbb460dc None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:21.264232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c7f77c-4445-4aaf-b50d-b0f5fbb460dc None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:21.266197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 441/874] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:21.271230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398299d-00a8-4a7c-b678-99bd29b02dca None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:21.271669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398299d-00a8-4a7c-b678-99bd29b02dca None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:21.271989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398299d-00a8-4a7c-b678-99bd29b02dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:21.272103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf835d-759e-4178-8210-d4c0bac74e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:21.272624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf835d-759e-4178-8210-d4c0bac74e5c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:21.272957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf835d-759e-4178-8210-d4c0bac74e5c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:21.273235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf835d-759e-4178-8210-d4c0bac74e5c None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:21.273928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf835d-759e-4178-8210-d4c0bac74e5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:21.274231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf835d-759e-4178-8210-d4c0bac74e5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:21.275775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf835d-759e-4178-8210-d4c0bac74e5c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:21.288579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 434/875] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:21.290669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-4398299d-00a8-4a7c-b678-99bd29b02dca None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:21.295290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4398299d-00a8-4a7c-b678-99bd29b02dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:21.296189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf835d-759e-4178-8210-d4c0bac74e5c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:21.320675 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:21.327075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:21.366813 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 442/876] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:21 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:21.419875 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf835d-759e-4178-8210-d4c0bac74e5c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:21.456240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67ccd7-39fb-47d1-a589-cffd92767462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:21.456899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67ccd7-39fb-47d1-a589-cffd92767462 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:21.457104 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67ccd7-39fb-47d1-a589-cffd92767462 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:21.457369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67ccd7-39fb-47d1-a589-cffd92767462 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:21.458092 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67ccd7-39fb-47d1-a589-cffd92767462 None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=da66e763177c47e39b3475fc42ccc8f0) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:21.458395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67ccd7-39fb-47d1-a589-cffd92767462 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:21.459842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67ccd7-39fb-47d1-a589-cffd92767462 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:21.466269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 435/877] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:21 2022] GET /identity/v3/role_assignments?user.id=da66e763177c47e39b3475fc42ccc8f0&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 610 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:21.543429 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398299d-00a8-4a7c-b678-99bd29b02dca None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:21.652106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67ccd7-39fb-47d1-a589-cffd92767462 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:21.658307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398299d-00a8-4a7c-b678-99bd29b02dca None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:21.707625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff002a-2824-439d-81da-30e44e693a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:21.708268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff002a-2824-439d-81da-30e44e693a3f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:21.708546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff002a-2824-439d-81da-30e44e693a3f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:21.708816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff002a-2824-439d-81da-30e44e693a3f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:21.709406 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff002a-2824-439d-81da-30e44e693a3f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:21.710774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff002a-2824-439d-81da-30e44e693a3f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-e9ff002a-2824-439d-81da-30e44e693a3f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:21.720806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:21.721825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:21.722702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 443/878] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:21.728703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff002a-2824-439d-81da-30e44e693a3f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:21.759143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:21.766905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:21.790064 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67ccd7-39fb-47d1-a589-cffd92767462 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:21.838627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a2b6f-4ec4-48ee-a85b-00f2d62c9ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:21.841022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a2b6f-4ec4-48ee-a85b-00f2d62c9ffb None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:21.841505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a2b6f-4ec4-48ee-a85b-00f2d62c9ffb None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:21.842253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a2b6f-4ec4-48ee-a85b-00f2d62c9ffb None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:21.843366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a2b6f-4ec4-48ee-a85b-00f2d62c9ffb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:21.845653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a2b6f-4ec4-48ee-a85b-00f2d62c9ffb None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:21.853819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff002a-2824-439d-81da-30e44e693a3f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:21.892255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0059b-b71b-427d-a678-6da620bc1918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:21.892862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0059b-b71b-427d-a678-6da620bc1918 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:21.893137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0059b-b71b-427d-a678-6da620bc1918 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:21.893462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf0059b-b71b-427d-a678-6da620bc1918 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:21.894096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0059b-b71b-427d-a678-6da620bc1918 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:21.894425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0059b-b71b-427d-a678-6da620bc1918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:21.895895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf0059b-b71b-427d-a678-6da620bc1918 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:21.906029 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 444/879] 10.222.0.105 () {56 vars in 1127 bytes} [Wed Aug 24 04:42:21 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:21.907221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0059b-b71b-427d-a678-6da620bc1918 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:21.941154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 436/880] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:42:21 2022] POST /identity/v3/services => generated 270 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:22.022310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:22.034233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf0059b-b71b-427d-a678-6da620bc1918 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:22.084369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f89e3-504d-4cfe-9878-7b463e82eace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:22.084982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f89e3-504d-4cfe-9878-7b463e82eace None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:22.085260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f89e3-504d-4cfe-9878-7b463e82eace None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:22.085582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f89e3-504d-4cfe-9878-7b463e82eace None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:22.086162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f89e3-504d-4cfe-9878-7b463e82eace None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:22.086461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f89e3-504d-4cfe-9878-7b463e82eace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:22.087915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f89e3-504d-4cfe-9878-7b463e82eace None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:22.094030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 445/881] 10.222.0.105 () {56 vars in 1127 bytes} [Wed Aug 24 04:42:21 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:23.001995 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf934438-4556-448d-9b93-e412634e0f03 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:23.002549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf934438-4556-448d-9b93-e412634e0f03 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:23.002549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf934438-4556-448d-9b93-e412634e0f03 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:23.004097 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 437/882] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:23.009363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d77ce6-f0cd-4187-8bc8-222d6ea791fd None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:23.009752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d77ce6-f0cd-4187-8bc8-222d6ea791fd None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:23.009937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d77ce6-f0cd-4187-8bc8-222d6ea791fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:23.029460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-23d77ce6-f0cd-4187-8bc8-222d6ea791fd None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:23.030643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-23d77ce6-f0cd-4187-8bc8-222d6ea791fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:23.102905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 446/883] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:23 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:23.368842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf934438-4556-448d-9b93-e412634e0f03 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:23.483183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf934438-4556-448d-9b93-e412634e0f03 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:23.519208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971ebf8-6726-4c9b-a345-5417cf6fb9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:23.519838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971ebf8-6726-4c9b-a345-5417cf6fb9f8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:23.520109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971ebf8-6726-4c9b-a345-5417cf6fb9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:23.520370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971ebf8-6726-4c9b-a345-5417cf6fb9f8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:23.520943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971ebf8-6726-4c9b-a345-5417cf6fb9f8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:23.522286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971ebf8-6726-4c9b-a345-5417cf6fb9f8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-0971ebf8-6726-4c9b-a345-5417cf6fb9f8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:23.531636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:23.532665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:23.533505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 438/884] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:42:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:23.538804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d77ce6-f0cd-4187-8bc8-222d6ea791fd None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:23.556294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130781e6-0557-46d3-835d-61215935d268 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:23.556594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130781e6-0557-46d3-835d-61215935d268 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:23.556836 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130781e6-0557-46d3-835d-61215935d268 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:23.559116 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 439/885] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:23 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:23.564148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43427eb-8140-4bfb-8ee0-8ac47b75cf92 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:23.564430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43427eb-8140-4bfb-8ee0-8ac47b75cf92 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:23.564702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43427eb-8140-4bfb-8ee0-8ac47b75cf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:23.583798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e43427eb-8140-4bfb-8ee0-8ac47b75cf92 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:23.584990 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e43427eb-8140-4bfb-8ee0-8ac47b75cf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:23.588715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:23.638726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:23.664826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d77ce6-f0cd-4187-8bc8-222d6ea791fd None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:23.674048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 440/886] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:23 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:23.701362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f21aa-7429-4229-8f63-056fe5f01544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:23.702368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f21aa-7429-4229-8f63-056fe5f01544 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:23.702841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f21aa-7429-4229-8f63-056fe5f01544 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:23.703359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f21aa-7429-4229-8f63-056fe5f01544 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:23.704224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f21aa-7429-4229-8f63-056fe5f01544 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:23.704734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f21aa-7429-4229-8f63-056fe5f01544 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:23.706274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f21aa-7429-4229-8f63-056fe5f01544 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:23.711869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 447/887] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:42:23 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:23.718319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43427eb-8140-4bfb-8ee0-8ac47b75cf92 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:23.820246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43427eb-8140-4bfb-8ee0-8ac47b75cf92 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:23.855784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312556e-ea55-492c-8b6b-fef5ea7555dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:23.856757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312556e-ea55-492c-8b6b-fef5ea7555dc None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:23.857227 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312556e-ea55-492c-8b6b-fef5ea7555dc None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:23.857687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312556e-ea55-492c-8b6b-fef5ea7555dc None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:23.858498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312556e-ea55-492c-8b6b-fef5ea7555dc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:23.859010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312556e-ea55-492c-8b6b-fef5ea7555dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:23.860679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312556e-ea55-492c-8b6b-fef5ea7555dc None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:23.866037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 441/888] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:42:23 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:23.872703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f21aa-7429-4229-8f63-056fe5f01544 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:23.960232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312556e-ea55-492c-8b6b-fef5ea7555dc None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:23.965355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:23.986525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f21aa-7429-4229-8f63-056fe5f01544 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:24.024904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14a8a5-de48-4873-8532-01e23aec8c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:24.025538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14a8a5-de48-4873-8532-01e23aec8c93 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.025776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14a8a5-de48-4873-8532-01e23aec8c93 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.026049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14a8a5-de48-4873-8532-01e23aec8c93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.026949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14a8a5-de48-4873-8532-01e23aec8c93 None admin] RBAC: Adding query filter params (interface=public, service_id=2caadf20e450446cb7e8d67774fef8cb, region_id=RegionOne) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:24.027304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14a8a5-de48-4873-8532-01e23aec8c93 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:24.028855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14a8a5-de48-4873-8532-01e23aec8c93 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:24.038697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 448/889] 10.222.0.105 () {56 vars in 1256 bytes} [Wed Aug 24 04:42:23 2022] GET /identity/v3/endpoints?service_id=2caadf20e450446cb7e8d67774fef8cb&interface=public®ion_id=RegionOne => generated 195 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:24.038697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225d17d-b850-451c-8abf-f9fd19cd3451 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.038697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225d17d-b850-451c-8abf-f9fd19cd3451 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.038697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e225d17d-b850-451c-8abf-f9fd19cd3451 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.038697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 449/890] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:24 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:24.039873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225d17d-b850-451c-8abf-f9fd19cd3451 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:24.052658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:24.074254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312556e-ea55-492c-8b6b-fef5ea7555dc None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:24.109752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d67a2f-d6e1-4787-b7ee-05a6a521f89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:24.110492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d67a2f-d6e1-4787-b7ee-05a6a521f89b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.110773 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d67a2f-d6e1-4787-b7ee-05a6a521f89b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.111067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d67a2f-d6e1-4787-b7ee-05a6a521f89b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.121809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d67a2f-d6e1-4787-b7ee-05a6a521f89b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-f3d67a2f-d6e1-4787-b7ee-05a6a521f89b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:24.126901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:24.127974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:24.129423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 442/891] 10.222.0.105 () {56 vars in 1125 bytes} [Wed Aug 24 04:42:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:24.131606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c601e-1c99-4aa8-8565-b35bd1c82080 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.131896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c601e-1c99-4aa8-8565-b35bd1c82080 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.132173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c601e-1c99-4aa8-8565-b35bd1c82080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.145765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e225d17d-b850-451c-8abf-f9fd19cd3451 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:24.149350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-286c601e-1c99-4aa8-8565-b35bd1c82080 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:24.150524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-286c601e-1c99-4aa8-8565-b35bd1c82080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:24.180977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78995b3d-3433-43f1-ad0c-5f3ab202317c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:24.181587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78995b3d-3433-43f1-ad0c-5f3ab202317c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.181867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78995b3d-3433-43f1-ad0c-5f3ab202317c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.182139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78995b3d-3433-43f1-ad0c-5f3ab202317c None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.182736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78995b3d-3433-43f1-ad0c-5f3ab202317c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:24.184916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78995b3d-3433-43f1-ad0c-5f3ab202317c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:24.190831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 450/892] 10.222.0.105 () {56 vars in 1092 bytes} [Wed Aug 24 04:42:24 2022] GET /identity/v3/services => generated 1274 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:42:24.192317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e988a-467c-411a-89ac-60bacfbb113b None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.192604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e988a-467c-411a-89ac-60bacfbb113b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.192881 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e988a-467c-411a-89ac-60bacfbb113b None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.194250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 451/893] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:24.194831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e988a-467c-411a-89ac-60bacfbb113b None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:24.220127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:24.230446 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 443/894] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:24 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:24.232233 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a76d8c-eac9-47e3-8f18-3b1b72b5ef8f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.232750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a76d8c-eac9-47e3-8f18-3b1b72b5ef8f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.233220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a76d8c-eac9-47e3-8f18-3b1b72b5ef8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.261589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c3a76d8c-eac9-47e3-8f18-3b1b72b5ef8f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:24.263391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c3a76d8c-eac9-47e3-8f18-3b1b72b5ef8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:24.290172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:24.326244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e988a-467c-411a-89ac-60bacfbb113b None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:24.356708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 444/895] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:24 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:24.368963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4618e-1321-4394-8ad9-066d272683df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:24.369570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4618e-1321-4394-8ad9-066d272683df None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.369845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4618e-1321-4394-8ad9-066d272683df None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.370112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4618e-1321-4394-8ad9-066d272683df None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.371385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4618e-1321-4394-8ad9-066d272683df None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:24.371751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4618e-1321-4394-8ad9-066d272683df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:24.373026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c4618e-1321-4394-8ad9-066d272683df None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:24.382181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 452/896] 10.222.0.105 () {56 vars in 1120 bytes} [Wed Aug 24 04:42:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:24.389074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a76d8c-eac9-47e3-8f18-3b1b72b5ef8f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:24.491212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a76d8c-eac9-47e3-8f18-3b1b72b5ef8f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:24.515614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:24.523476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4618e-1321-4394-8ad9-066d272683df None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:24.528701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a052c-f365-40b2-8b33-c86878f00f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:24.529351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a052c-f365-40b2-8b33-c86878f00f1f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.536675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a052c-f365-40b2-8b33-c86878f00f1f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.536675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a052c-f365-40b2-8b33-c86878f00f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.540868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a052c-f365-40b2-8b33-c86878f00f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:24.543233 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a052c-f365-40b2-8b33-c86878f00f1f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-2d2a052c-f365-40b2-8b33-c86878f00f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:24.560368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:24.561563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:24.562816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:24.562816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:24.562816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:24.562816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 445/897] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:24.567895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a052c-f365-40b2-8b33-c86878f00f1f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:24.650125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4618e-1321-4394-8ad9-066d272683df None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:24.684955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133572d-20c2-44ff-b424-ed55b8a5ce8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:24.685774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133572d-20c2-44ff-b424-ed55b8a5ce8a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.686102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a052c-f365-40b2-8b33-c86878f00f1f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:24.686205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133572d-20c2-44ff-b424-ed55b8a5ce8a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.686591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5133572d-20c2-44ff-b424-ed55b8a5ce8a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.687307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133572d-20c2-44ff-b424-ed55b8a5ce8a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:24.688771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5133572d-20c2-44ff-b424-ed55b8a5ce8a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:24.690702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-5133572d-20c2-44ff-b424-ed55b8a5ce8a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:24.698240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:24.699877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 04:42:24.700892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:24.704862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 453/898] 10.222.0.105 () {56 vars in 1137 bytes} [Wed Aug 24 04:42:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:24.705700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133572d-20c2-44ff-b424-ed55b8a5ce8a None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:24.726592 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d27ac7-3b50-4412-97cd-396f08236063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:24.727468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d27ac7-3b50-4412-97cd-396f08236063 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.728371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d27ac7-3b50-4412-97cd-396f08236063 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.728804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d27ac7-3b50-4412-97cd-396f08236063 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.729911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d27ac7-3b50-4412-97cd-396f08236063 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:24.730503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d27ac7-3b50-4412-97cd-396f08236063 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:24.732736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d27ac7-3b50-4412-97cd-396f08236063 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:24.752657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 446/899] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:24.752657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d27ac7-3b50-4412-97cd-396f08236063 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:24.788760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:24.817010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5133572d-20c2-44ff-b424-ed55b8a5ce8a None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:24.852557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba6960-8f0f-4f6e-adc0-ce0534de9028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:24.853187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba6960-8f0f-4f6e-adc0-ce0534de9028 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.853466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba6960-8f0f-4f6e-adc0-ce0534de9028 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.853748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba6960-8f0f-4f6e-adc0-ce0534de9028 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.854367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba6960-8f0f-4f6e-adc0-ce0534de9028 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:24.855873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d27ac7-3b50-4412-97cd-396f08236063 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:24.856062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba6960-8f0f-4f6e-adc0-ce0534de9028 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:24.890040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e5997-0820-4a4d-b893-d9cde66a7db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:24.890646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e5997-0820-4a4d-b893-d9cde66a7db8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:24.890924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e5997-0820-4a4d-b893-d9cde66a7db8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:24.891222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e5997-0820-4a4d-b893-d9cde66a7db8 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:24.891844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e5997-0820-4a4d-b893-d9cde66a7db8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:24.892168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e5997-0820-4a4d-b893-d9cde66a7db8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:24.893525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3e5997-0820-4a4d-b893-d9cde66a7db8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:24.898147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 447/900] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:24 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:24.898912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e5997-0820-4a4d-b893-d9cde66a7db8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:24.928619 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 454/901] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:42:24 2022] POST /identity/v3/services => generated 242 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:24.929694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba6960-8f0f-4f6e-adc0-ce0534de9028 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:25.000326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:25.040860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e5997-0820-4a4d-b893-d9cde66a7db8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:25.044027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:25.065022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba6960-8f0f-4f6e-adc0-ce0534de9028 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:25.082843 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1518b43-9a91-4d3d-a026-a7c7b7735933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:25.083567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1518b43-9a91-4d3d-a026-a7c7b7735933 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:25.083799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1518b43-9a91-4d3d-a026-a7c7b7735933 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:25.084071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1518b43-9a91-4d3d-a026-a7c7b7735933 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:25.093118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1518b43-9a91-4d3d-a026-a7c7b7735933 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:25.094625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1518b43-9a91-4d3d-a026-a7c7b7735933 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:25.107612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d1f1d-3463-4ee4-957c-c8d47eedcb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:25.108219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d1f1d-3463-4ee4-957c-c8d47eedcb62 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:25.108492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d1f1d-3463-4ee4-957c-c8d47eedcb62 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-c1518b43-9a91-4d3d-a026-a7c7b7735933 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:25.108753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:25.109656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:25.110601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:25.110601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:25.110601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:25.110601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:25.110601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:25.110601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:25.110601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:25.110601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:25.110601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:25.110601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:25.111167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d1f1d-3463-4ee4-957c-c8d47eedcb62 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:25.111167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d1f1d-3463-4ee4-957c-c8d47eedcb62 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:25.111167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d1f1d-3463-4ee4-957c-c8d47eedcb62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:25.114733 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 448/902] 10.222.0.105 () {56 vars in 1122 bytes} [Wed Aug 24 04:42:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:25.114818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d1f1d-3463-4ee4-957c-c8d47eedcb62 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:25.117454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1518b43-9a91-4d3d-a026-a7c7b7735933 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:25.118208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 455/903] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:24 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:25.125391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d1f1d-3463-4ee4-957c-c8d47eedcb62 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:25.257429 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d1f1d-3463-4ee4-957c-c8d47eedcb62 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:25.259066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1518b43-9a91-4d3d-a026-a7c7b7735933 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:25.276746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:25.297279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ceff-370f-4694-b453-a497c0a8b50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:25.297279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ceff-370f-4694-b453-a497c0a8b50b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:25.297279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ceff-370f-4694-b453-a497c0a8b50b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:25.297279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7ceff-370f-4694-b453-a497c0a8b50b None admin] PATH_INFO: `/v3/endpoints` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:25.297712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7ceff-370f-4694-b453-a497c0a8b50b None admin] RBAC: Adding query filter params (interface=public, service_id=524382d2a08f48da81d48fec47aaaac6, region_id=RegionOne) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:25.297712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7ceff-370f-4694-b453-a497c0a8b50b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:25.298358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7ceff-370f-4694-b453-a497c0a8b50b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:25.303983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 456/904] 10.222.0.105 () {56 vars in 1256 bytes} [Wed Aug 24 04:42:25 2022] GET /identity/v3/endpoints?service_id=524382d2a08f48da81d48fec47aaaac6&interface=public®ion_id=RegionOne => generated 195 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:25.310089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ceff-370f-4694-b453-a497c0a8b50b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:25.312284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f71f-f7c4-4780-9451-10a0dbba94b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:25.313343 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f71f-f7c4-4780-9451-10a0dbba94b5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:25.314656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f71f-f7c4-4780-9451-10a0dbba94b5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:25.315196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8f71f-f7c4-4780-9451-10a0dbba94b5 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:25.316394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f71f-f7c4-4780-9451-10a0dbba94b5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:25.316903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f71f-f7c4-4780-9451-10a0dbba94b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:25.318617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8f71f-f7c4-4780-9451-10a0dbba94b5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:25.343589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 449/905] 10.222.0.105 () {56 vars in 1154 bytes} [Wed Aug 24 04:42:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:25.344712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f71f-f7c4-4780-9451-10a0dbba94b5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:25.368451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:25.446599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7ceff-370f-4694-b453-a497c0a8b50b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:25.477555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8f71f-f7c4-4780-9451-10a0dbba94b5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:25.498362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed720b-b9bb-412b-8c75-19b1ad0da0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:25.498969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed720b-b9bb-412b-8c75-19b1ad0da0bb None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:25.499275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed720b-b9bb-412b-8c75-19b1ad0da0bb None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:25.499546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ed720b-b9bb-412b-8c75-19b1ad0da0bb None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:25.500146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed720b-b9bb-412b-8c75-19b1ad0da0bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:25.502019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ed720b-b9bb-412b-8c75-19b1ad0da0bb None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:25.510684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 457/906] 10.222.0.105 () {56 vars in 1092 bytes} [Wed Aug 24 04:42:25 2022] GET /identity/v3/services => generated 1504 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:42:25.517029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01447801-fd78-4ac4-9daa-a18f9f1d6bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:25.517780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01447801-fd78-4ac4-9daa-a18f9f1d6bd1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:25.518132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01447801-fd78-4ac4-9daa-a18f9f1d6bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:25.518477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01447801-fd78-4ac4-9daa-a18f9f1d6bd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:25.601705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01447801-fd78-4ac4-9daa-a18f9f1d6bd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:25.603234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01447801-fd78-4ac4-9daa-a18f9f1d6bd1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:25.623698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:25.713792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 450/907] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:25 2022] GET /identity/v3/auth/tokens => generated 1476 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:42:25.720645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed720b-b9bb-412b-8c75-19b1ad0da0bb None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:25.769289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:25.825796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ed720b-b9bb-412b-8c75-19b1ad0da0bb None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:25.866179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64843456-bf84-409b-981e-03a17c364526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:25.867162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64843456-bf84-409b-981e-03a17c364526 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:25.867669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64843456-bf84-409b-981e-03a17c364526 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:25.868141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64843456-bf84-409b-981e-03a17c364526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:25.879809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64843456-bf84-409b-981e-03a17c364526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:25.881571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64843456-bf84-409b-981e-03a17c364526 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-64843456-bf84-409b-981e-03a17c364526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:25.896563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:25.897555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:25.898479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:25.898479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:25.898479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:25.900610 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 458/908] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:25.909406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01447801-fd78-4ac4-9daa-a18f9f1d6bd1 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:26.009387 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01447801-fd78-4ac4-9daa-a18f9f1d6bd1 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:26.027449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:26.046107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ec07a-333b-43a4-a0c6-1bdb3734dc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:26.046923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ec07a-333b-43a4-a0c6-1bdb3734dc18 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:26.047386 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ec07a-333b-43a4-a0c6-1bdb3734dc18 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:26.047780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ec07a-333b-43a4-a0c6-1bdb3734dc18 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:26.048555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ec07a-333b-43a4-a0c6-1bdb3734dc18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:26.048989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ec07a-333b-43a4-a0c6-1bdb3734dc18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:26.050470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538ec07a-333b-43a4-a0c6-1bdb3734dc18 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:26.063241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 451/909] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:26.072254 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64843456-bf84-409b-981e-03a17c364526 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:26.181157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:26.186136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64843456-bf84-409b-981e-03a17c364526 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:26.225087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e212125-a076-4764-bc67-3fa44120cc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:26.226114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e212125-a076-4764-bc67-3fa44120cc8a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:26.226600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e212125-a076-4764-bc67-3fa44120cc8a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:26.227114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e212125-a076-4764-bc67-3fa44120cc8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:26.234713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e212125-a076-4764-bc67-3fa44120cc8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:26.236423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e212125-a076-4764-bc67-3fa44120cc8a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-0e212125-a076-4764-bc67-3fa44120cc8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:26.247930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:26.248820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:26.249743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:26.249743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:26.249743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:26.249743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:26.249743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:26.249743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:26.249743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:26.249743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:26.252119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 459/910] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:26.263037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ec07a-333b-43a4-a0c6-1bdb3734dc18 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:26.325879 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbb5d8-4554-4990-b425-ff0772c70b9f None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:26.326232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbb5d8-4554-4990-b425-ff0772c70b9f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:26.326425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbb5d8-4554-4990-b425-ff0772c70b9f None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:26.328023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 460/911] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:26.333059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30223d-dd28-409f-81af-9f251e6f7a38 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:26.333415 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30223d-dd28-409f-81af-9f251e6f7a38 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:26.333691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30223d-dd28-409f-81af-9f251e6f7a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:26.352126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-3f30223d-dd28-409f-81af-9f251e6f7a38 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:26.353674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3f30223d-dd28-409f-81af-9f251e6f7a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:26.383866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ec07a-333b-43a4-a0c6-1bdb3734dc18 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:26.435278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 461/912] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:26 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:26.459126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f21510-b993-4bce-85f9-e7dbfb54d8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:26.459795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f21510-b993-4bce-85f9-e7dbfb54d8c3 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:26.460069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f21510-b993-4bce-85f9-e7dbfb54d8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:26.460340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f21510-b993-4bce-85f9-e7dbfb54d8c3 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:26.461028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f21510-b993-4bce-85f9-e7dbfb54d8c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:26.461326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f21510-b993-4bce-85f9-e7dbfb54d8c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:26.462789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f21510-b993-4bce-85f9-e7dbfb54d8c3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:26.519119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 452/913] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:26.531046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30223d-dd28-409f-81af-9f251e6f7a38 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:26.700446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30223d-dd28-409f-81af-9f251e6f7a38 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:26.715414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:26.740240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155261fa-3113-42e0-9b63-0664ee347570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:26.744856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155261fa-3113-42e0-9b63-0664ee347570 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:26.744856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155261fa-3113-42e0-9b63-0664ee347570 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:26.744856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155261fa-3113-42e0-9b63-0664ee347570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:26.745163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f21510-b993-4bce-85f9-e7dbfb54d8c3 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:26.745289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155261fa-3113-42e0-9b63-0664ee347570 None admin] RBAC: Adding query filter params (role.id=dced619d94db43ed8cccb9c80be594b9, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=da66e763177c47e39b3475fc42ccc8f0) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:26.745289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155261fa-3113-42e0-9b63-0664ee347570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:26.745289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155261fa-3113-42e0-9b63-0664ee347570 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:26.751903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 462/914] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:26 2022] GET /identity/v3/role_assignments?user.id=da66e763177c47e39b3475fc42ccc8f0&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=dced619d94db43ed8cccb9c80be594b9 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:26.802966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:26.913634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f21510-b993-4bce-85f9-e7dbfb54d8c3 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:26.959122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f9031-a3c6-42ab-98ed-2d9ece620be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:26.959122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f9031-a3c6-42ab-98ed-2d9ece620be5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:26.959122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f9031-a3c6-42ab-98ed-2d9ece620be5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:26.959122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f9031-a3c6-42ab-98ed-2d9ece620be5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:26.959513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f9031-a3c6-42ab-98ed-2d9ece620be5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:26.959513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f9031-a3c6-42ab-98ed-2d9ece620be5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-e35f9031-a3c6-42ab-98ed-2d9ece620be5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:26.975044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:26.976286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 04:42:26.977400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:26.980485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 453/915] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:42:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:26.987514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155261fa-3113-42e0-9b63-0664ee347570 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:27.129244 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b53939-b0b7-485f-a4e8-d138f2b17238 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:27.129546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b53939-b0b7-485f-a4e8-d138f2b17238 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:27.129824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b53939-b0b7-485f-a4e8-d138f2b17238 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:27.135389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 454/916] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:27.137655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b2da5-5347-42fb-b94f-c813376b70fb None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:27.138037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b2da5-5347-42fb-b94f-c813376b70fb None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:27.138383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b2da5-5347-42fb-b94f-c813376b70fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:27.159083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-448b2da5-5347-42fb-b94f-c813376b70fb None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:27.161003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-448b2da5-5347-42fb-b94f-c813376b70fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:27.173198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155261fa-3113-42e0-9b63-0664ee347570 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:27.217022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8109db1-d966-4b8f-b980-4a5e4a433a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:27.217557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8109db1-d966-4b8f-b980-4a5e4a433a41 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:27.217838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8109db1-d966-4b8f-b980-4a5e4a433a41 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:27.218113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8109db1-d966-4b8f-b980-4a5e4a433a41 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:27.218747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8109db1-d966-4b8f-b980-4a5e4a433a41 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:27.219184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8109db1-d966-4b8f-b980-4a5e4a433a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:27.221023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8109db1-d966-4b8f-b980-4a5e4a433a41 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:27.226044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 463/917] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:42:26 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:27.231818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8109db1-d966-4b8f-b980-4a5e4a433a41 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:27.251784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:27.253055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 455/918] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:27 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:27.358202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8109db1-d966-4b8f-b980-4a5e4a433a41 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:27.403989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa735b-7bfa-4193-8255-45a0b76f9525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:27.404615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa735b-7bfa-4193-8255-45a0b76f9525 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:27.404891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa735b-7bfa-4193-8255-45a0b76f9525 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:27.405162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa735b-7bfa-4193-8255-45a0b76f9525 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:27.405804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa735b-7bfa-4193-8255-45a0b76f9525 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:27.406105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa735b-7bfa-4193-8255-45a0b76f9525 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:27.407936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa735b-7bfa-4193-8255-45a0b76f9525 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:27.413915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 464/919] 10.222.0.105 () {56 vars in 1119 bytes} [Wed Aug 24 04:42:27 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:27.419655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b2da5-5347-42fb-b94f-c813376b70fb None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:27.452289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:27.538345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b2da5-5347-42fb-b94f-c813376b70fb None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:27.558527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa735b-7bfa-4193-8255-45a0b76f9525 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:27.581472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e8eae-51df-430c-83df-c8e5f7fdc156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:27.582179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8eae-51df-430c-83df-c8e5f7fdc156 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:27.582462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8eae-51df-430c-83df-c8e5f7fdc156 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:27.582731 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8eae-51df-430c-83df-c8e5f7fdc156 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:27.583549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e8eae-51df-430c-83df-c8e5f7fdc156 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:27.584914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e8eae-51df-430c-83df-c8e5f7fdc156 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:27.651145 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:27.662926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 456/920] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:42:27 2022] POST /identity/v3/endpoints => generated 348 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:27.681433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa735b-7bfa-4193-8255-45a0b76f9525 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:27.688290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3d0fb-cf99-4f8d-af27-dac849b36067 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:27.688584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3d0fb-cf99-4f8d-af27-dac849b36067 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:27.688840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3d0fb-cf99-4f8d-af27-dac849b36067 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:27.690799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 457/921] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:27.696471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12767e-31a8-4d28-90bb-7b65b6f91a29 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:27.705700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12767e-31a8-4d28-90bb-7b65b6f91a29 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:27.705700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12767e-31a8-4d28-90bb-7b65b6f91a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:27.724050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ba12767e-31a8-4d28-90bb-7b65b6f91a29 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:27.725383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ba12767e-31a8-4d28-90bb-7b65b6f91a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:27.738368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc95931-7e76-46b3-ab0f-d71edb0e4896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:27.739325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc95931-7e76-46b3-ab0f-d71edb0e4896 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:27.739616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc95931-7e76-46b3-ab0f-d71edb0e4896 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:27.740683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc95931-7e76-46b3-ab0f-d71edb0e4896 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:27.741304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc95931-7e76-46b3-ab0f-d71edb0e4896 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:27.743395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc95931-7e76-46b3-ab0f-d71edb0e4896 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-6dc95931-7e76-46b3-ab0f-d71edb0e4896 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:27.756585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:27.757721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:27.758615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:27.763128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 465/922] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:27.765548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc95931-7e76-46b3-ab0f-d71edb0e4896 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:27.824250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 458/923] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:27 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:27.902725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc95931-7e76-46b3-ab0f-d71edb0e4896 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:27.938057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a12b28-8bdd-4996-8391-204d8b66af1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:27.938671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12b28-8bdd-4996-8391-204d8b66af1e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:27.938950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12b28-8bdd-4996-8391-204d8b66af1e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:27.939252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a12b28-8bdd-4996-8391-204d8b66af1e None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:27.939946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a12b28-8bdd-4996-8391-204d8b66af1e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:27.940249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a12b28-8bdd-4996-8391-204d8b66af1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:27.941924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a12b28-8bdd-4996-8391-204d8b66af1e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:27.947039 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 466/924] 10.222.0.105 () {56 vars in 1127 bytes} [Wed Aug 24 04:42:27 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:27.953047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12767e-31a8-4d28-90bb-7b65b6f91a29 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:27.981872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:28.070751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12767e-31a8-4d28-90bb-7b65b6f91a29 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:28.111192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5049ed-4b81-4201-b182-aa9ab50d2fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:28.112157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5049ed-4b81-4201-b182-aa9ab50d2fa6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:28.112649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5049ed-4b81-4201-b182-aa9ab50d2fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:28.113147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5049ed-4b81-4201-b182-aa9ab50d2fa6 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:28.114056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5049ed-4b81-4201-b182-aa9ab50d2fa6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:28.114562 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5049ed-4b81-4201-b182-aa9ab50d2fa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:28.116262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5049ed-4b81-4201-b182-aa9ab50d2fa6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:28.126170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 459/925] 10.222.0.105 () {56 vars in 1127 bytes} [Wed Aug 24 04:42:27 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:28.128052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a12b28-8bdd-4996-8391-204d8b66af1e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:28.131868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5049ed-4b81-4201-b182-aa9ab50d2fa6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:28.132606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:28.231179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:28.251255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5049ed-4b81-4201-b182-aa9ab50d2fa6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:28.267154 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a12b28-8bdd-4996-8391-204d8b66af1e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:28.289119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19422c1c-fd31-431e-9f3e-5dd451f2cb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:28.289903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19422c1c-fd31-431e-9f3e-5dd451f2cb71 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:28.290291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19422c1c-fd31-431e-9f3e-5dd451f2cb71 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:28.290658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19422c1c-fd31-431e-9f3e-5dd451f2cb71 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:28.291384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19422c1c-fd31-431e-9f3e-5dd451f2cb71 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:28.293564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19422c1c-fd31-431e-9f3e-5dd451f2cb71 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-19422c1c-fd31-431e-9f3e-5dd451f2cb71 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:28.302664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:28.303869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 04:42:28.304965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:28.311695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 460/926] 10.222.0.105 () {56 vars in 1137 bytes} [Wed Aug 24 04:42:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:28.312600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19422c1c-fd31-431e-9f3e-5dd451f2cb71 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:28.316756 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d775ca-b765-4e49-8468-f190f42c4ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:28.318502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d775ca-b765-4e49-8468-f190f42c4ad9 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:28.318975 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d775ca-b765-4e49-8468-f190f42c4ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:28.319456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d775ca-b765-4e49-8468-f190f42c4ad9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:28.320330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d775ca-b765-4e49-8468-f190f42c4ad9 None admin] RBAC: Adding query filter params (interface=public, service_id=309c608051d844ed8d8b5af12ea609df, region_id=RegionOne) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:28.320830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d775ca-b765-4e49-8468-f190f42c4ad9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:28.322362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d775ca-b765-4e49-8468-f190f42c4ad9 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:28.335349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 467/927] 10.222.0.105 () {56 vars in 1256 bytes} [Wed Aug 24 04:42:28 2022] GET /identity/v3/endpoints?service_id=309c608051d844ed8d8b5af12ea609df&interface=public®ion_id=RegionOne => generated 195 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:28.337797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d775ca-b765-4e49-8468-f190f42c4ad9 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:28.431247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:28.433790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19422c1c-fd31-431e-9f3e-5dd451f2cb71 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:28.469759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07922e-1ee5-4955-a44b-9018aff7b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:28.470739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07922e-1ee5-4955-a44b-9018aff7b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:28.471042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07922e-1ee5-4955-a44b-9018aff7b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:28.471312 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07922e-1ee5-4955-a44b-9018aff7b9f0 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:28.472476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07922e-1ee5-4955-a44b-9018aff7b9f0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:28.472779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07922e-1ee5-4955-a44b-9018aff7b9f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:28.474322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f07922e-1ee5-4955-a44b-9018aff7b9f0 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:28.479960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d775ca-b765-4e49-8468-f190f42c4ad9 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:28.484877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 461/928] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:28 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:28.485858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07922e-1ee5-4955-a44b-9018aff7b9f0 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:28.500340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:28.526723 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafcc07-2a9f-42ff-aebb-83c7f7b9e25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:28.527777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafcc07-2a9f-42ff-aebb-83c7f7b9e25b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:28.528338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafcc07-2a9f-42ff-aebb-83c7f7b9e25b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:28.528824 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daafcc07-2a9f-42ff-aebb-83c7f7b9e25b None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:28.529648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafcc07-2a9f-42ff-aebb-83c7f7b9e25b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:28.531712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daafcc07-2a9f-42ff-aebb-83c7f7b9e25b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:28.547626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 468/929] 10.222.0.105 () {56 vars in 1092 bytes} [Wed Aug 24 04:42:28 2022] GET /identity/v3/services => generated 1504 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:42:28.600543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07922e-1ee5-4955-a44b-9018aff7b9f0 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:28.641912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e467a3-2a6f-4726-95f0-c3751866164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:28.642534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e467a3-2a6f-4726-95f0-c3751866164d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:28.642859 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e467a3-2a6f-4726-95f0-c3751866164d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:28.643223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e467a3-2a6f-4726-95f0-c3751866164d None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:28.643920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e467a3-2a6f-4726-95f0-c3751866164d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:28.644224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e467a3-2a6f-4726-95f0-c3751866164d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:28.645643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e467a3-2a6f-4726-95f0-c3751866164d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:28.652370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 462/930] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:28 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:28.658237 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafcc07-2a9f-42ff-aebb-83c7f7b9e25b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:28.775487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daafcc07-2a9f-42ff-aebb-83c7f7b9e25b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:28.807045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:28.812608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12023919-fbe0-4b6f-a7a4-5f63bbcf4fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:28.813222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12023919-fbe0-4b6f-a7a4-5f63bbcf4fe0 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:28.813497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12023919-fbe0-4b6f-a7a4-5f63bbcf4fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:28.813764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12023919-fbe0-4b6f-a7a4-5f63bbcf4fe0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:28.814501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12023919-fbe0-4b6f-a7a4-5f63bbcf4fe0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:28.816146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12023919-fbe0-4b6f-a7a4-5f63bbcf4fe0 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:28.868876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407a339-609c-481d-b626-70aed0deeb39 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:28.869165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407a339-609c-481d-b626-70aed0deeb39 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:28.869588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407a339-609c-481d-b626-70aed0deeb39 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:28.871321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 463/931] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:28.876117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb391a-46ac-4ea6-b0fa-9fb9bc88fc67 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:28.876399 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb391a-46ac-4ea6-b0fa-9fb9bc88fc67 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:28.876669 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb391a-46ac-4ea6-b0fa-9fb9bc88fc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:28.895044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-defb391a-46ac-4ea6-b0fa-9fb9bc88fc67 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:28.896285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-defb391a-46ac-4ea6-b0fa-9fb9bc88fc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:28.936191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 469/932] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:42:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:28.977778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:28.989535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 464/933] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:28 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:29.266409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12023919-fbe0-4b6f-a7a4-5f63bbcf4fe0 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:29.373222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12023919-fbe0-4b6f-a7a4-5f63bbcf4fe0 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:29.409167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e23b3-889b-4b2d-943f-5d80422a653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:29.409805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e23b3-889b-4b2d-943f-5d80422a653f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:29.410138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e23b3-889b-4b2d-943f-5d80422a653f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:29.410413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e23b3-889b-4b2d-943f-5d80422a653f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:29.421218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e23b3-889b-4b2d-943f-5d80422a653f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-1d1e23b3-889b-4b2d-943f-5d80422a653f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:29.426461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:29.427413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:29.428490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 470/934] 10.222.0.105 () {56 vars in 1125 bytes} [Wed Aug 24 04:42:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:29.439944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb391a-46ac-4ea6-b0fa-9fb9bc88fc67 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:29.549009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb391a-46ac-4ea6-b0fa-9fb9bc88fc67 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:29.585991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fddc5-986c-4898-b244-822782e54328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:29.586930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fddc5-986c-4898-b244-822782e54328 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:29.587424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fddc5-986c-4898-b244-822782e54328 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:29.587883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fddc5-986c-4898-b244-822782e54328 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:29.588920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fddc5-986c-4898-b244-822782e54328 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:29.589592 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fddc5-986c-4898-b244-822782e54328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:29.591210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fddc5-986c-4898-b244-822782e54328 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:29.604891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 465/935] 10.222.0.105 () {56 vars in 1120 bytes} [Wed Aug 24 04:42:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:29.608089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e23b3-889b-4b2d-943f-5d80422a653f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:29.661659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:29.678895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25f363-0a2c-4440-9dab-4dbe87102228 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:29.679242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25f363-0a2c-4440-9dab-4dbe87102228 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:29.679461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25f363-0a2c-4440-9dab-4dbe87102228 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:29.680952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 466/936] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:29.686635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75b6cb-44bc-4d68-a72a-ed49443a1552 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:29.689077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75b6cb-44bc-4d68-a72a-ed49443a1552 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:29.689077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75b6cb-44bc-4d68-a72a-ed49443a1552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:29.706960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7f75b6cb-44bc-4d68-a72a-ed49443a1552 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:29.708505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7f75b6cb-44bc-4d68-a72a-ed49443a1552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:29.724614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e23b3-889b-4b2d-943f-5d80422a653f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:29.772925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01ff99-9d3c-4840-ab5f-65283e8a2a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:29.773550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01ff99-9d3c-4840-ab5f-65283e8a2a99 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:29.773821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01ff99-9d3c-4840-ab5f-65283e8a2a99 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:29.774086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01ff99-9d3c-4840-ab5f-65283e8a2a99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:29.780283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:29.789481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01ff99-9d3c-4840-ab5f-65283e8a2a99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:29.790981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d01ff99-9d3c-4840-ab5f-65283e8a2a99 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:29.796242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 467/937] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:29 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-5d01ff99-9d3c-4840-ab5f-65283e8a2a99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:29.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:29.814278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:29.815500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:29.815500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:29.815500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:29.821896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 471/938] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:29.824429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75b6cb-44bc-4d68-a72a-ed49443a1552 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:29.927947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75b6cb-44bc-4d68-a72a-ed49443a1552 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:29.965543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf757e-4008-4028-9cae-8f769af205d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:29.966416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf757e-4008-4028-9cae-8f769af205d4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:29.966856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf757e-4008-4028-9cae-8f769af205d4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:29.967289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf757e-4008-4028-9cae-8f769af205d4 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:29.968060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf757e-4008-4028-9cae-8f769af205d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:29.968548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf757e-4008-4028-9cae-8f769af205d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:29.970078 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf757e-4008-4028-9cae-8f769af205d4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:29.986980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 468/939] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:29.988603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01ff99-9d3c-4840-ab5f-65283e8a2a99 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:30.094793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:30.098491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf757e-4008-4028-9cae-8f769af205d4 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:30.109645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01ff99-9d3c-4840-ab5f-65283e8a2a99 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:30.154632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40da293-e374-495f-9f7a-1fa3957f2307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:30.155292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40da293-e374-495f-9f7a-1fa3957f2307 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.155572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40da293-e374-495f-9f7a-1fa3957f2307 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.155993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40da293-e374-495f-9f7a-1fa3957f2307 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.164563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40da293-e374-495f-9f7a-1fa3957f2307 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:30.164890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:30.166542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40da293-e374-495f-9f7a-1fa3957f2307 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b40da293-e374-495f-9f7a-1fa3957f2307 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:30.184818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:30.185848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:30.186859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:30.186859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:30.186859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:30.186859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:30.186859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:30.186859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:30.186859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:30.186859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:30.186859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:30.186859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:30.191312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 472/940] 10.222.0.105 () {56 vars in 1122 bytes} [Wed Aug 24 04:42:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:30.193060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40da293-e374-495f-9f7a-1fa3957f2307 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:30.239057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf757e-4008-4028-9cae-8f769af205d4 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:30.284948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:30.285864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.286315 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.286776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.287637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:30.289403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:30.293797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: WARNING keystone.server.flask.common [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 04:42:30.319804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40da293-e374-495f-9f7a-1fa3957f2307 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:30.350272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.sql.core [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 04:42:30.350272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 04:42:30.350272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [parameters: {'id': '8e5b8365def340f6b5abecc8fc69c6e0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 04:42:30.350272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82574) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 04:42:30.356422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f013bf2-3379-48dd-9802-8f09f6216b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:30.357118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f013bf2-3379-48dd-9802-8f09f6216b09 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.357458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f013bf2-3379-48dd-9802-8f09f6216b09 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.357819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f013bf2-3379-48dd-9802-8f09f6216b09 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.358662 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f013bf2-3379-48dd-9802-8f09f6216b09 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:30.359048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f013bf2-3379-48dd-9802-8f09f6216b09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:30.360622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f013bf2-3379-48dd-9802-8f09f6216b09 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8e5b8365def340f6b5abecc8fc69c6e0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8e5b8365def340f6b5abecc8fc69c6e0. Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application conn.query(q) Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 04:42:30.373298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application result.read() Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 04:42:30.374166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application next(self.gen) Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application yield resource Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application next(self.gen) Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise self.value Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application session.commit() Aug 24 04:42:30.375128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self.session.flush() Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application compat.raise_( Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 04:42:30.376036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 04:42:30.376927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application conn.query(q) Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application result.read() Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 04:42:30.377713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [parameters: {'id': '8e5b8365def340f6b5abecc8fc69c6e0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:30.378605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:30.380123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:30.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:30.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 04:42:30.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 04:42:30.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:30.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:30.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 04:42:30.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 04:42:30.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8e5b8365def340f6b5abecc8fc69c6e0. Aug 24 04:42:30.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:30.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 469/941] 10.222.0.105 () {60 vars in 1145 bytes} [Wed Aug 24 04:42:30 2022] POST /identity/v3/projects => generated 262 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:30.388622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:30.391709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 473/942] 10.222.0.105 () {56 vars in 1154 bytes} [Wed Aug 24 04:42:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:30.398710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f013bf2-3379-48dd-9802-8f09f6216b09 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:30.429174 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:30.527296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68894d-b5dd-4eac-b058-eea99673bcfb None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:30.530538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f013bf2-3379-48dd-9802-8f09f6216b09 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:30.541095 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:30.570746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa01189-87d5-4f2b-9bd9-b3a9bc0d4c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:30.571440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa01189-87d5-4f2b-9bd9-b3a9bc0d4c14 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.571720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa01189-87d5-4f2b-9bd9-b3a9bc0d4c14 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.571992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa01189-87d5-4f2b-9bd9-b3a9bc0d4c14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.576415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013cc4c-0cb9-4be9-8f68-01d8fd9a71e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:30.577534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013cc4c-0cb9-4be9-8f68-01d8fd9a71e6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.578080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013cc4c-0cb9-4be9-8f68-01d8fd9a71e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.578696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013cc4c-0cb9-4be9-8f68-01d8fd9a71e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.585936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa01189-87d5-4f2b-9bd9-b3a9bc0d4c14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:30.587712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa01189-87d5-4f2b-9bd9-b3a9bc0d4c14 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:30.589297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013cc4c-0cb9-4be9-8f68-01d8fd9a71e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:30.591682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013cc4c-0cb9-4be9-8f68-01d8fd9a71e6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-dfa01189-87d5-4f2b-9bd9-b3a9bc0d4c14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:30.605195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:30.606130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:30.607494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:30.607494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:30.607494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:30.607494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 474/943] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:30.607831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bb837-36ef-4859-a611-d7a21ad142e7 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.608047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bb837-36ef-4859-a611-d7a21ad142e7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.608317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bb837-36ef-4859-a611-d7a21ad142e7 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.609644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 475/944] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:30 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:30.612952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bb837-36ef-4859-a611-d7a21ad142e7 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-4013cc4c-0cb9-4be9-8f68-01d8fd9a71e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:30.614964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:30.616036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:30.617137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:30.617137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:30.617137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:30.617137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:30.617137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:30.617137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:30.617137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:30.617137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:30.622397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 470/945] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:30.623834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601999e1-6800-4d25-acbb-86ba947fbfb8 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.624122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601999e1-6800-4d25-acbb-86ba947fbfb8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.624396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601999e1-6800-4d25-acbb-86ba947fbfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.643284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-601999e1-6800-4d25-acbb-86ba947fbfb8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:30.644810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-601999e1-6800-4d25-acbb-86ba947fbfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:30.692116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:30.719404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 471/946] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:30 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:30.719404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601999e1-6800-4d25-acbb-86ba947fbfb8 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:30.729426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bb837-36ef-4859-a611-d7a21ad142e7 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:30.771256 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be558a-1fb5-4299-bb41-62fe8b143f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:30.771966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be558a-1fb5-4299-bb41-62fe8b143f20 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.772311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be558a-1fb5-4299-bb41-62fe8b143f20 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.772648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be558a-1fb5-4299-bb41-62fe8b143f20 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.773781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be558a-1fb5-4299-bb41-62fe8b143f20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:30.774164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be558a-1fb5-4299-bb41-62fe8b143f20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:30.775629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be558a-1fb5-4299-bb41-62fe8b143f20 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:30.794686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 476/947] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:30.797903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be558a-1fb5-4299-bb41-62fe8b143f20 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:30.858962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:30.864951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601999e1-6800-4d25-acbb-86ba947fbfb8 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:30.907218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e9060-c42c-4302-b9e4-d397a426f684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:30.908096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9060-c42c-4302-b9e4-d397a426f684 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.908545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9060-c42c-4302-b9e4-d397a426f684 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.908814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9060-c42c-4302-b9e4-d397a426f684 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.909464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e9060-c42c-4302-b9e4-d397a426f684 None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:30.909759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e9060-c42c-4302-b9e4-d397a426f684 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:30.911094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be558a-1fb5-4299-bb41-62fe8b143f20 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:30.911219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e9060-c42c-4302-b9e4-d397a426f684 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:30.935093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 472/948] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:30 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:30.935093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15666f-db90-4671-ba80-dce57b46e9bc None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.935093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15666f-db90-4671-ba80-dce57b46e9bc None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.935093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15666f-db90-4671-ba80-dce57b46e9bc None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.936240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 473/949] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:30.945117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c59f5-71f9-493a-93fd-5c5f36975b9c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.945400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c59f5-71f9-493a-93fd-5c5f36975b9c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.945672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c59f5-71f9-493a-93fd-5c5f36975b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.953567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:30.960294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c35f10-4745-4130-b45c-da354d62eac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:30.961478 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c35f10-4745-4130-b45c-da354d62eac5 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:30.962023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c35f10-4745-4130-b45c-da354d62eac5 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:30.962515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c35f10-4745-4130-b45c-da354d62eac5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:30.966415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ff0c59f5-71f9-493a-93fd-5c5f36975b9c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:30.967714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ff0c59f5-71f9-493a-93fd-5c5f36975b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:30.971610 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c35f10-4745-4130-b45c-da354d62eac5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:30.973450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c35f10-4745-4130-b45c-da354d62eac5 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-07c35f10-4745-4130-b45c-da354d62eac5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:30.987421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:30.988399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:30.989325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:30.989325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:30.989325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:30.989325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:30.989325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:30.989325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:30.989325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:30.989325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:30.997013 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 477/950] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:30.997013 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c35f10-4745-4130-b45c-da354d62eac5 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:31.043587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 474/951] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:30 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:31.052501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c59f5-71f9-493a-93fd-5c5f36975b9c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:31.121642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c35f10-4745-4130-b45c-da354d62eac5 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:31.164396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b92a9-243d-41e9-9080-fcfd56c1c073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:31.165032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b92a9-243d-41e9-9080-fcfd56c1c073 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:31.165425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b92a9-243d-41e9-9080-fcfd56c1c073 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:31.165701 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b92a9-243d-41e9-9080-fcfd56c1c073 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:31.166385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b92a9-243d-41e9-9080-fcfd56c1c073 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:31.168437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b92a9-243d-41e9-9080-fcfd56c1c073 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:31.177156 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b82b92a9-243d-41e9-9080-fcfd56c1c073 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:31.179986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:31.180976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:31.183541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 478/952] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:31.187686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b92a9-243d-41e9-9080-fcfd56c1c073 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:31.202822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c59f5-71f9-493a-93fd-5c5f36975b9c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:31.240185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47b44-b688-4dfa-a4e9-8ccd9d8a09bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:31.240849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47b44-b688-4dfa-a4e9-8ccd9d8a09bc None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:31.241127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47b44-b688-4dfa-a4e9-8ccd9d8a09bc None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:31.241394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47b44-b688-4dfa-a4e9-8ccd9d8a09bc None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:31.242071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47b44-b688-4dfa-a4e9-8ccd9d8a09bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:31.242367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47b44-b688-4dfa-a4e9-8ccd9d8a09bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:31.243866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47b44-b688-4dfa-a4e9-8ccd9d8a09bc None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:31.257193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 475/953] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:31.263946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47b44-b688-4dfa-a4e9-8ccd9d8a09bc None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:31.264430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:31.323425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b92a9-243d-41e9-9080-fcfd56c1c073 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:31.360865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601bfa-d78e-4728-87e2-16f412e722a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:31.361477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601bfa-d78e-4728-87e2-16f412e722a4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:31.361744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601bfa-d78e-4728-87e2-16f412e722a4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:31.362016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62601bfa-d78e-4728-87e2-16f412e722a4 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:31.362644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62601bfa-d78e-4728-87e2-16f412e722a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:31.362944 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62601bfa-d78e-4728-87e2-16f412e722a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:31.364330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62601bfa-d78e-4728-87e2-16f412e722a4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:31.368942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 479/954] 10.222.0.105 () {56 vars in 1127 bytes} [Wed Aug 24 04:42:31 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:31.370385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601bfa-d78e-4728-87e2-16f412e722a4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:31.376795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47b44-b688-4dfa-a4e9-8ccd9d8a09bc None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:31.412694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44681e80-5e3e-41cf-8d9d-93e03b185a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:31.413350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681e80-5e3e-41cf-8d9d-93e03b185a0f None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:31.413620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681e80-5e3e-41cf-8d9d-93e03b185a0f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:31.413894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44681e80-5e3e-41cf-8d9d-93e03b185a0f None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/da66e763177c47e39b3475fc42ccc8f0/roles/dced619d94db43ed8cccb9c80be594b9` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:31.416084 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44681e80-5e3e-41cf-8d9d-93e03b185a0f None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=da66e763177c47e39b3475fc42ccc8f0, role_id=dced619d94db43ed8cccb9c80be594b9)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:31.418203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44681e80-5e3e-41cf-8d9d-93e03b185a0f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:31.472511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62601bfa-d78e-4728-87e2-16f412e722a4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:31.482608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 476/955] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:42:31 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/da66e763177c47e39b3475fc42ccc8f0/roles/dced619d94db43ed8cccb9c80be594b9 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:42:31.483401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44681e80-5e3e-41cf-8d9d-93e03b185a0f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:31.496721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:31.506531 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:31.526999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc8b0f-30a5-4376-a078-11833d1ac9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:31.527891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc8b0f-30a5-4376-a078-11833d1ac9c8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:31.529090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc8b0f-30a5-4376-a078-11833d1ac9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:31.529090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc8b0f-30a5-4376-a078-11833d1ac9c8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:31.529957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc8b0f-30a5-4376-a078-11833d1ac9c8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:31.531435 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc8b0f-30a5-4376-a078-11833d1ac9c8 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-06fc8b0f-30a5-4376-a078-11833d1ac9c8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:31.542656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:31.543869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 04:42:31.544734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:31.549266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 480/956] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:31.550711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc8b0f-30a5-4376-a078-11833d1ac9c8 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:31.630302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44681e80-5e3e-41cf-8d9d-93e03b185a0f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:31.667765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc8b0f-30a5-4376-a078-11833d1ac9c8 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:31.670036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5f1f3-d326-4066-9664-ad5108c09472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:31.671644 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5f1f3-d326-4066-9664-ad5108c09472 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:31.672066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5f1f3-d326-4066-9664-ad5108c09472 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:31.672497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f5f1f3-d326-4066-9664-ad5108c09472 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:31.673282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5f1f3-d326-4066-9664-ad5108c09472 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:31.673808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5f1f3-d326-4066-9664-ad5108c09472 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:31.675373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f5f1f3-d326-4066-9664-ad5108c09472 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:31.683091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 477/957] 10.222.0.105 () {56 vars in 1127 bytes} [Wed Aug 24 04:42:31 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:31.689045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5f1f3-d326-4066-9664-ad5108c09472 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:31.706399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819a8d2-366d-4897-9213-fa056ba64182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:31.707296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819a8d2-366d-4897-9213-fa056ba64182 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:31.707733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819a8d2-366d-4897-9213-fa056ba64182 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:31.708277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819a8d2-366d-4897-9213-fa056ba64182 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:31.709181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819a8d2-366d-4897-9213-fa056ba64182 None admin] RBAC: Adding query filter params (name=compute) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:31.709627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819a8d2-366d-4897-9213-fa056ba64182 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:31.711143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819a8d2-366d-4897-9213-fa056ba64182 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:31.717773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 481/958] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:31 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:31.722995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819a8d2-366d-4897-9213-fa056ba64182 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:31.772224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:31.785322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:31.826353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f5f1f3-d326-4066-9664-ad5108c09472 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:31.831092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819a8d2-366d-4897-9213-fa056ba64182 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:31.867124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfcac9-900f-418d-a472-dbd8865b528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:31.867851 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfcac9-900f-418d-a472-dbd8865b528f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:31.868193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfcac9-900f-418d-a472-dbd8865b528f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:31.868535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfcac9-900f-418d-a472-dbd8865b528f None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:31.869357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfcac9-900f-418d-a472-dbd8865b528f None admin] RBAC: Adding query filter params (type=compute) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:31.869849 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfcac9-900f-418d-a472-dbd8865b528f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:31.871654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfcac9-900f-418d-a472-dbd8865b528f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:31.873228 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3fd61-7827-4a62-bf64-f640aa09958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:31.874522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3fd61-7827-4a62-bf64-f640aa09958a None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:31.874949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3fd61-7827-4a62-bf64-f640aa09958a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:31.875634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3fd61-7827-4a62-bf64-f640aa09958a None admin] PATH_INFO: `/v3/endpoints` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:31.876467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3fd61-7827-4a62-bf64-f640aa09958a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:31.878029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3fd61-7827-4a62-bf64-f640aa09958a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:31.883359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 482/959] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:31 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:31.979796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 478/960] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:42:31 2022] POST /identity/v3/endpoints => generated 351 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:42:33.028869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78202b77-47b9-4c40-bb85-50f41a6d87e4 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:33.029426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78202b77-47b9-4c40-bb85-50f41a6d87e4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:33.029426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78202b77-47b9-4c40-bb85-50f41a6d87e4 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:33.031052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 483/961] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:33.036951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681364-1b92-415f-bd6c-517553e037d5 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:33.037365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681364-1b92-415f-bd6c-517553e037d5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:33.044675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15681364-1b92-415f-bd6c-517553e037d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:33.058015 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-15681364-1b92-415f-bd6c-517553e037d5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:33.059236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-15681364-1b92-415f-bd6c-517553e037d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:33.140622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 479/962] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:33 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:33.416759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78202b77-47b9-4c40-bb85-50f41a6d87e4 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:33.531042 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78202b77-47b9-4c40-bb85-50f41a6d87e4 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:33.539969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d4fdd-dc08-4b2e-8c7a-3a5d8971001c None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:33.540256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d4fdd-dc08-4b2e-8c7a-3a5d8971001c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:33.540532 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d4fdd-dc08-4b2e-8c7a-3a5d8971001c None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:33.544160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:33.546964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 480/963] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:33.548731 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e8463-e36a-4056-a38f-fc5227d8a512 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:33.548972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e8463-e36a-4056-a38f-fc5227d8a512 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:33.549247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e8463-e36a-4056-a38f-fc5227d8a512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:33.568099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c76e8463-e36a-4056-a38f-fc5227d8a512 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:33.569286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c76e8463-e36a-4056-a38f-fc5227d8a512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:33.574239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d170dba-2a7a-4581-901e-2fefa77cd9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:33.575157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d170dba-2a7a-4581-901e-2fefa77cd9fc None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:33.575640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d170dba-2a7a-4581-901e-2fefa77cd9fc None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:33.576123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d170dba-2a7a-4581-901e-2fefa77cd9fc None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:33.576983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d170dba-2a7a-4581-901e-2fefa77cd9fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:33.579931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d170dba-2a7a-4581-901e-2fefa77cd9fc None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:33.583961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WARNING keystone.server.flask.common [None req-2d170dba-2a7a-4581-901e-2fefa77cd9fc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 04:42:33.624100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:33.645199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.sql.core [None req-2d170dba-2a7a-4581-901e-2fefa77cd9fc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 04:42:33.645199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 04:42:33.645199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [parameters: {'user_id': '0bb6821c65d8466e9cf50d0a0961134f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 04:42:33.645199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=82575) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 04:42:33.651172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 481/964] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:33 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-2d170dba-2a7a-4581-901e-2fefa77cd9fc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application conn.query(q) Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application result.read() Aug 24 04:42:33.653657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 04:42:33.654559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application next(self.gen) Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application yield resource Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application next(self.gen) Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise self.value Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application session.commit() Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 04:42:33.663605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self.session.flush() Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application compat.raise_( Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 04:42:33.664505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 04:42:33.665401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application conn.query(q) Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application result.read() Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 04:42:33.667336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [parameters: {'user_id': '0bb6821c65d8466e9cf50d0a0961134f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:33.670032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 04:42:33.671969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 04:42:33.674125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 04:42:33.674125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 04:42:33.674125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 04:42:33.674125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 04:42:33.674125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 04:42:33.674125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:33.674125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 484/965] 10.222.0.105 () {60 vars in 1136 bytes} [Wed Aug 24 04:42:33 2022] POST /identity/v3/users => generated 160 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:33.675110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e8463-e36a-4056-a38f-fc5227d8a512 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:33.777655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e8463-e36a-4056-a38f-fc5227d8a512 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:33.813974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14069fe1-9fec-420a-8f7e-44cba87366c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:33.815287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14069fe1-9fec-420a-8f7e-44cba87366c5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:33.815566 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14069fe1-9fec-420a-8f7e-44cba87366c5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:33.815832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14069fe1-9fec-420a-8f7e-44cba87366c5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:33.816797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceeb14-f825-478c-a98a-eaa8967f8058 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:33.817086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceeb14-f825-478c-a98a-eaa8967f8058 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:33.817361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceeb14-f825-478c-a98a-eaa8967f8058 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:33.824056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14069fe1-9fec-420a-8f7e-44cba87366c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:33.825585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14069fe1-9fec-420a-8f7e-44cba87366c5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:33.827366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 485/966] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:33 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:33.829650 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39581ee-b7ff-4497-8d41-ff6431855503 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:33.829939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39581ee-b7ff-4497-8d41-ff6431855503 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:33.830216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39581ee-b7ff-4497-8d41-ff6431855503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-14069fe1-9fec-420a-8f7e-44cba87366c5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:33.838804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:33.840039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:33.841044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:33.841044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:33.841044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:33.841044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:33.841044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:33.841044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:33.841044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:33.841044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:33.841044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 04:42:33.841044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:33.844699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 482/967] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:42:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:33.845929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14069fe1-9fec-420a-8f7e-44cba87366c5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:33.853194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d39581ee-b7ff-4497-8d41-ff6431855503 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:33.855741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d39581ee-b7ff-4497-8d41-ff6431855503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:33.893604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:33.947325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 486/968] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:33 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:33.953891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39581ee-b7ff-4497-8d41-ff6431855503 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:33.966792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14069fe1-9fec-420a-8f7e-44cba87366c5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:34.007625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7ec37-e33b-4238-8ff0-8533e6bdfb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:34.008241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7ec37-e33b-4238-8ff0-8533e6bdfb30 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.008519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7ec37-e33b-4238-8ff0-8533e6bdfb30 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.008797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7ec37-e33b-4238-8ff0-8533e6bdfb30 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.009495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7ec37-e33b-4238-8ff0-8533e6bdfb30 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:34.009825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7ec37-e33b-4238-8ff0-8533e6bdfb30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:34.011266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7ec37-e33b-4238-8ff0-8533e6bdfb30 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:34.054619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 483/969] 10.222.0.105 () {56 vars in 1106 bytes} [Wed Aug 24 04:42:33 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:34.054619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48241f24-b37c-4cf0-a92d-aa2ae4c196ae None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.054619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48241f24-b37c-4cf0-a92d-aa2ae4c196ae None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.054619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48241f24-b37c-4cf0-a92d-aa2ae4c196ae None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.054619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 484/970] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:34.056679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06033459-53bf-4978-8748-3f8e070205c1 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.058000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06033459-53bf-4978-8748-3f8e070205c1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.058000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06033459-53bf-4978-8748-3f8e070205c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.070915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39581ee-b7ff-4497-8d41-ff6431855503 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:34.073368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:34.077902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-06033459-53bf-4978-8748-3f8e070205c1 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:34.079256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-06033459-53bf-4978-8748-3f8e070205c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:34.119661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171c19c-322b-466c-9fd3-fbbc8dafc7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:34.120594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171c19c-322b-466c-9fd3-fbbc8dafc7ad None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.121373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171c19c-322b-466c-9fd3-fbbc8dafc7ad None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.121961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171c19c-322b-466c-9fd3-fbbc8dafc7ad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.137047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171c19c-322b-466c-9fd3-fbbc8dafc7ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-9171c19c-322b-466c-9fd3-fbbc8dafc7ad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:34.143733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:34.144708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:34.145550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 487/971] 10.222.0.105 () {56 vars in 1125 bytes} [Wed Aug 24 04:42:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:34.152055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171c19c-322b-466c-9fd3-fbbc8dafc7ad None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:34.164707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 485/972] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:34 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:34.224181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06033459-53bf-4978-8748-3f8e070205c1 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:34.261261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:34.280693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171c19c-322b-466c-9fd3-fbbc8dafc7ad None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:34.331954 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8829537-2424-4033-b453-67da0ff16540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:34.332563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8829537-2424-4033-b453-67da0ff16540 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.332837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8829537-2424-4033-b453-67da0ff16540 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.333099 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8829537-2424-4033-b453-67da0ff16540 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.333759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8829537-2424-4033-b453-67da0ff16540 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:34.334055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8829537-2424-4033-b453-67da0ff16540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:34.336069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8829537-2424-4033-b453-67da0ff16540 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:34.346710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 488/973] 10.222.0.105 () {56 vars in 1120 bytes} [Wed Aug 24 04:42:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:34.352291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8829537-2424-4033-b453-67da0ff16540 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:34.365414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06033459-53bf-4978-8748-3f8e070205c1 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:34.388989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:34.403337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024be5d-a750-4ddf-ae2f-5cb460a24e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:34.404613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024be5d-a750-4ddf-ae2f-5cb460a24e0a None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.404613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024be5d-a750-4ddf-ae2f-5cb460a24e0a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.404613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024be5d-a750-4ddf-ae2f-5cb460a24e0a None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.405211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024be5d-a750-4ddf-ae2f-5cb460a24e0a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:34.406574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024be5d-a750-4ddf-ae2f-5cb460a24e0a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:34.465964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8829537-2424-4033-b453-67da0ff16540 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:34.483488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 486/974] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:42:34 2022] POST /identity/v3/services => generated 243 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:34.483488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024be5d-a750-4ddf-ae2f-5cb460a24e0a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:34.512106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71420fc-17eb-4176-bfc6-7b5708101089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:34.512755 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71420fc-17eb-4176-bfc6-7b5708101089 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.513031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71420fc-17eb-4176-bfc6-7b5708101089 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.513295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71420fc-17eb-4176-bfc6-7b5708101089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.525132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71420fc-17eb-4176-bfc6-7b5708101089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:34.526826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71420fc-17eb-4176-bfc6-7b5708101089 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-f71420fc-17eb-4176-bfc6-7b5708101089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:34.543641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:34.544882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:34.546135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:34.546135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:34.546135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:34.546135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 489/975] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:34.550798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71420fc-17eb-4176-bfc6-7b5708101089 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:34.595439 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:34.605405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024be5d-a750-4ddf-ae2f-5cb460a24e0a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:34.635161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:34.643268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6511d4-9d07-4e14-a832-a69f82870267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:34.644127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6511d4-9d07-4e14-a832-a69f82870267 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.644406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6511d4-9d07-4e14-a832-a69f82870267 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.644669 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6511d4-9d07-4e14-a832-a69f82870267 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.645239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6511d4-9d07-4e14-a832-a69f82870267 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:34.646654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6511d4-9d07-4e14-a832-a69f82870267 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-fd6511d4-9d07-4e14-a832-a69f82870267 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:34.659440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:34.660569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:34.661435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 487/976] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:34.666619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6511d4-9d07-4e14-a832-a69f82870267 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:34.675477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71420fc-17eb-4176-bfc6-7b5708101089 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:34.715919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e5300-46ae-43d9-8b8a-00a0727d8158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:34.716670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e5300-46ae-43d9-8b8a-00a0727d8158 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.717054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e5300-46ae-43d9-8b8a-00a0727d8158 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.717424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e5300-46ae-43d9-8b8a-00a0727d8158 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.718186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e5300-46ae-43d9-8b8a-00a0727d8158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:34.718588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e5300-46ae-43d9-8b8a-00a0727d8158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:34.720084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e5300-46ae-43d9-8b8a-00a0727d8158 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:34.733185 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 490/977] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:34.738689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e5300-46ae-43d9-8b8a-00a0727d8158 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:34.776354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6511d4-9d07-4e14-a832-a69f82870267 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:34.811475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bb1bd-d7c0-45eb-8ac3-b97664a1037a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:34.812087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bb1bd-d7c0-45eb-8ac3-b97664a1037a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.812365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bb1bd-d7c0-45eb-8ac3-b97664a1037a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.812645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bb1bd-d7c0-45eb-8ac3-b97664a1037a None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.813264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bb1bd-d7c0-45eb-8ac3-b97664a1037a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:34.813563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bb1bd-d7c0-45eb-8ac3-b97664a1037a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:34.815184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189bb1bd-d7c0-45eb-8ac3-b97664a1037a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:34.820394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 488/978] 10.222.0.105 () {56 vars in 1127 bytes} [Wed Aug 24 04:42:34 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:34.826190 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bb1bd-d7c0-45eb-8ac3-b97664a1037a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:34.846299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e5300-46ae-43d9-8b8a-00a0727d8158 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:34.849977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:34.875919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:34.880637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9552330-e614-4b69-86ba-7718cd53a7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:34.881260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9552330-e614-4b69-86ba-7718cd53a7cf None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.881536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9552330-e614-4b69-86ba-7718cd53a7cf None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.881802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9552330-e614-4b69-86ba-7718cd53a7cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.889138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9552330-e614-4b69-86ba-7718cd53a7cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:34.890578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9552330-e614-4b69-86ba-7718cd53a7cf None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-c9552330-e614-4b69-86ba-7718cd53a7cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:34.902670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:34.904022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:34.905180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 491/979] 10.222.0.105 () {56 vars in 1122 bytes} [Wed Aug 24 04:42:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:34.909421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9552330-e614-4b69-86ba-7718cd53a7cf None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:34.940183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bb1bd-d7c0-45eb-8ac3-b97664a1037a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:34.975863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af52f5c-1d8c-4504-938c-0beaac24ed8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:34.976880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af52f5c-1d8c-4504-938c-0beaac24ed8f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:34.977153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af52f5c-1d8c-4504-938c-0beaac24ed8f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:34.977419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af52f5c-1d8c-4504-938c-0beaac24ed8f None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:34.978043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af52f5c-1d8c-4504-938c-0beaac24ed8f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:34.978344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af52f5c-1d8c-4504-938c-0beaac24ed8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:34.980755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af52f5c-1d8c-4504-938c-0beaac24ed8f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:34.985608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 489/980] 10.222.0.105 () {56 vars in 1127 bytes} [Wed Aug 24 04:42:34 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:34.991391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af52f5c-1d8c-4504-938c-0beaac24ed8f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:35.032707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9552330-e614-4b69-86ba-7718cd53a7cf None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:35.069882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8d1db-47f2-4ab5-9909-3a236197d3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:35.070589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8d1db-47f2-4ab5-9909-3a236197d3d3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:35.070874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8d1db-47f2-4ab5-9909-3a236197d3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:35.071190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8d1db-47f2-4ab5-9909-3a236197d3d3 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:35.071925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8d1db-47f2-4ab5-9909-3a236197d3d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:35.072795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8d1db-47f2-4ab5-9909-3a236197d3d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:35.074288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8d1db-47f2-4ab5-9909-3a236197d3d3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:35.080923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:35.093290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 492/981] 10.222.0.105 () {56 vars in 1154 bytes} [Wed Aug 24 04:42:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:35.098788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af52f5c-1d8c-4504-938c-0beaac24ed8f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:35.098944 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8d1db-47f2-4ab5-9909-3a236197d3d3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:35.134364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ad69d-ab86-4b51-826f-531ba408c885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:35.135761 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ad69d-ab86-4b51-826f-531ba408c885 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:35.136036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ad69d-ab86-4b51-826f-531ba408c885 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:35.136303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ad69d-ab86-4b51-826f-531ba408c885 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:35.137007 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ad69d-ab86-4b51-826f-531ba408c885 None admin] RBAC: Adding query filter params (interface=admin, service_id=309c608051d844ed8d8b5af12ea609df, region_id=RegionOne) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:35.137309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ad69d-ab86-4b51-826f-531ba408c885 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:35.138659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ad69d-ab86-4b51-826f-531ba408c885 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:35.143979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 490/982] 10.222.0.105 () {56 vars in 1254 bytes} [Wed Aug 24 04:42:34 2022] GET /identity/v3/endpoints?service_id=309c608051d844ed8d8b5af12ea609df&interface=admin®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:35.149164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ad69d-ab86-4b51-826f-531ba408c885 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:35.153984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:35.211541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8d1db-47f2-4ab5-9909-3a236197d3d3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:35.246718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b9ef2-4d48-48f7-a18c-639a68641875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:35.247655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9ef2-4d48-48f7-a18c-639a68641875 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:35.247957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9ef2-4d48-48f7-a18c-639a68641875 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:35.248230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9ef2-4d48-48f7-a18c-639a68641875 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:35.258968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ad69d-ab86-4b51-826f-531ba408c885 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:35.295596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66523f65-e23c-49bb-a17c-085b696d5758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:35.296205 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523f65-e23c-49bb-a17c-085b696d5758 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:35.296482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523f65-e23c-49bb-a17c-085b696d5758 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:35.297109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66523f65-e23c-49bb-a17c-085b696d5758 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:35.297718 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66523f65-e23c-49bb-a17c-085b696d5758 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:35.299560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66523f65-e23c-49bb-a17c-085b696d5758 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:35.311055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 491/983] 10.222.0.105 () {56 vars in 1092 bytes} [Wed Aug 24 04:42:35 2022] GET /identity/v3/services => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:42:35.330744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b9ef2-4d48-48f7-a18c-639a68641875 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:35.332252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b9ef2-4d48-48f7-a18c-639a68641875 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:35.388602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:35.444009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 493/984] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:35 2022] GET /identity/v3/auth/tokens => generated 1572 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 04:42:35.450452 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66523f65-e23c-49bb-a17c-085b696d5758 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:35.466912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:35.557400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66523f65-e23c-49bb-a17c-085b696d5758 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:35.591977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd043bea-40ae-4e58-b268-520ab1282a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:35.592611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd043bea-40ae-4e58-b268-520ab1282a94 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:35.592888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd043bea-40ae-4e58-b268-520ab1282a94 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:35.593155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd043bea-40ae-4e58-b268-520ab1282a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:35.604378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd043bea-40ae-4e58-b268-520ab1282a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:35.605865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd043bea-40ae-4e58-b268-520ab1282a94 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-cd043bea-40ae-4e58-b268-520ab1282a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:35.622523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:35.623567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:35.625139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:35.625139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:35.625139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:35.625493 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 492/985] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:35.629501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b9ef2-4d48-48f7-a18c-639a68641875 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:35.729201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b9ef2-4d48-48f7-a18c-639a68641875 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:35.763589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55703bf8-840f-441d-b4c2-323663456c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:35.764255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55703bf8-840f-441d-b4c2-323663456c29 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:35.764539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55703bf8-840f-441d-b4c2-323663456c29 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:35.764833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55703bf8-840f-441d-b4c2-323663456c29 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:35.765474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55703bf8-840f-441d-b4c2-323663456c29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:35.765774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55703bf8-840f-441d-b4c2-323663456c29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:35.767168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55703bf8-840f-441d-b4c2-323663456c29 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:35.779164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 494/986] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:35.784978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd043bea-40ae-4e58-b268-520ab1282a94 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:35.850381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:35.889184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd043bea-40ae-4e58-b268-520ab1282a94 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:35.922786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c3253-ce80-42a4-905c-2aeeddc44c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:35.923455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c3253-ce80-42a4-905c-2aeeddc44c03 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:35.923732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c3253-ce80-42a4-905c-2aeeddc44c03 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:35.924002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5c3253-ce80-42a4-905c-2aeeddc44c03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:35.930638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c3253-ce80-42a4-905c-2aeeddc44c03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:35.932834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5c3253-ce80-42a4-905c-2aeeddc44c03 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-5c5c3253-ce80-42a4-905c-2aeeddc44c03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:35.944017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:35.944911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:35.945829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:35.945829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:35.945829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:35.945829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:35.945829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:35.945829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:35.945829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:35.945829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:35.947344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 493/987] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:35.950981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55703bf8-840f-441d-b4c2-323663456c29 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:35.964198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:36.050882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55703bf8-840f-441d-b4c2-323663456c29 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:36.085089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c1918-e925-4a23-9ba2-b2fd18a1f2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:36.085735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c1918-e925-4a23-9ba2-b2fd18a1f2e8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:36.085949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c1918-e925-4a23-9ba2-b2fd18a1f2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:36.086220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c1918-e925-4a23-9ba2-b2fd18a1f2e8 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:36.086898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c1918-e925-4a23-9ba2-b2fd18a1f2e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:36.087228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c1918-e925-4a23-9ba2-b2fd18a1f2e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:36.088839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c1918-e925-4a23-9ba2-b2fd18a1f2e8 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:36.101079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 495/988] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:36.107573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c3253-ce80-42a4-905c-2aeeddc44c03 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:36.216290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5c3253-ce80-42a4-905c-2aeeddc44c03 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:36.239812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:36.250792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ec4920-748d-4ebe-8f79-19243b85f441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:36.251436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec4920-748d-4ebe-8f79-19243b85f441 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:36.251753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec4920-748d-4ebe-8f79-19243b85f441 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:36.262802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ec4920-748d-4ebe-8f79-19243b85f441 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:36.262802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec4920-748d-4ebe-8f79-19243b85f441 None admin] RBAC: Adding query filter params (role.id=dced619d94db43ed8cccb9c80be594b9, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=da66e763177c47e39b3475fc42ccc8f0) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:36.262802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec4920-748d-4ebe-8f79-19243b85f441 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:36.262802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ec4920-748d-4ebe-8f79-19243b85f441 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:36.262802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 494/989] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:36 2022] GET /identity/v3/role_assignments?user.id=da66e763177c47e39b3475fc42ccc8f0&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=dced619d94db43ed8cccb9c80be594b9 => generated 610 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:36.456831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c21d6-32b9-4d2a-a97d-b56c5fdf6218 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:36.457043 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c21d6-32b9-4d2a-a97d-b56c5fdf6218 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:36.457349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c21d6-32b9-4d2a-a97d-b56c5fdf6218 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:36.458789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 496/990] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:36.464671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa576aa2-6fcf-4b40-9e2f-0ccb132091be None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:36.464960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa576aa2-6fcf-4b40-9e2f-0ccb132091be None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:36.465230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa576aa2-6fcf-4b40-9e2f-0ccb132091be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:36.484077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-aa576aa2-6fcf-4b40-9e2f-0ccb132091be None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:36.485314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-aa576aa2-6fcf-4b40-9e2f-0ccb132091be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:36.551566 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 495/991] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:36 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:36.825988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c21d6-32b9-4d2a-a97d-b56c5fdf6218 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:36.898036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:36.930160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c21d6-32b9-4d2a-a97d-b56c5fdf6218 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:36.966266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb0290-3ed0-462c-9e37-4a5f58adf2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:36.966889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb0290-3ed0-462c-9e37-4a5f58adf2d3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:36.967210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb0290-3ed0-462c-9e37-4a5f58adf2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:36.967484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fb0290-3ed0-462c-9e37-4a5f58adf2d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:36.968058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb0290-3ed0-462c-9e37-4a5f58adf2d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:36.969455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fb0290-3ed0-462c-9e37-4a5f58adf2d3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b6fb0290-3ed0-462c-9e37-4a5f58adf2d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:36.977957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:36.979153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:36.979999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 497/992] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:36.986647 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa576aa2-6fcf-4b40-9e2f-0ccb132091be None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:37.088156 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa576aa2-6fcf-4b40-9e2f-0ccb132091be None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:37.105593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:37.122860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a8309-1010-49fe-8ca6-01bcf881cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:37.123508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a8309-1010-49fe-8ca6-01bcf881cdae None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:37.123783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a8309-1010-49fe-8ca6-01bcf881cdae None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:37.124052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a8309-1010-49fe-8ca6-01bcf881cdae None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:37.124674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a8309-1010-49fe-8ca6-01bcf881cdae None admin] RBAC: Adding query filter params (name=compute) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:37.124971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a8309-1010-49fe-8ca6-01bcf881cdae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:37.126382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a8309-1010-49fe-8ca6-01bcf881cdae None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:37.136823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 496/993] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:36 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:37.137777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb0290-3ed0-462c-9e37-4a5f58adf2d3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:37.247060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c7cdf-8af0-442d-8afc-9cf6b590581e None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:37.247386 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c7cdf-8af0-442d-8afc-9cf6b590581e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:37.247620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c7cdf-8af0-442d-8afc-9cf6b590581e None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:37.249043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 497/994] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:37.252209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fb0290-3ed0-462c-9e37-4a5f58adf2d3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:37.254045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeaaf1-eb28-4709-9c55-48c9c7e81829 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:37.254325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeaaf1-eb28-4709-9c55-48c9c7e81829 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:37.254596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeeaaf1-eb28-4709-9c55-48c9c7e81829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:37.273922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6eeeaaf1-eb28-4709-9c55-48c9c7e81829 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:37.275286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6eeeaaf1-eb28-4709-9c55-48c9c7e81829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:37.289173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a21202-8184-4f01-bdb5-fba81330d22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:37.289998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a21202-8184-4f01-bdb5-fba81330d22f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:37.290136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a21202-8184-4f01-bdb5-fba81330d22f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:37.290315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a21202-8184-4f01-bdb5-fba81330d22f None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:37.291560 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a21202-8184-4f01-bdb5-fba81330d22f None admin] RBAC: Adding query filter params (type=compute) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:37.291934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a21202-8184-4f01-bdb5-fba81330d22f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:37.293221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a21202-8184-4f01-bdb5-fba81330d22f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:37.294831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:37.298631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 498/995] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:37 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:37.304657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a21202-8184-4f01-bdb5-fba81330d22f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:37.351578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 498/996] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:37 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:37.413903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a21202-8184-4f01-bdb5-fba81330d22f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:37.448306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb18504-1bec-450a-988b-4814832b2bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:37.448915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb18504-1bec-450a-988b-4814832b2bb5 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:37.449190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb18504-1bec-450a-988b-4814832b2bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:37.449457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb18504-1bec-450a-988b-4814832b2bb5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:37.450146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb18504-1bec-450a-988b-4814832b2bb5 None admin] RBAC: Adding query filter params (interface=public, service_id=c9984beafb144dadbe00efd32d49760d, region_id=RegionOne) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:37.450452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb18504-1bec-450a-988b-4814832b2bb5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:37.451835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb18504-1bec-450a-988b-4814832b2bb5 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:37.461114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 499/997] 10.222.0.105 () {56 vars in 1256 bytes} [Wed Aug 24 04:42:37 2022] GET /identity/v3/endpoints?service_id=c9984beafb144dadbe00efd32d49760d&interface=public®ion_id=RegionOne => generated 195 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:37.462421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeeaaf1-eb28-4709-9c55-48c9c7e81829 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:37.561848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeeaaf1-eb28-4709-9c55-48c9c7e81829 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:37.574856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:37.595124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315dd796-2522-4291-b68d-2fe132646ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:37.595729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dd796-2522-4291-b68d-2fe132646ef1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:37.606707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dd796-2522-4291-b68d-2fe132646ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:37.606707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315dd796-2522-4291-b68d-2fe132646ef1 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:37.606707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315dd796-2522-4291-b68d-2fe132646ef1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:37.606707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315dd796-2522-4291-b68d-2fe132646ef1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:37.606707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 499/998] 10.222.0.105 () {56 vars in 1092 bytes} [Wed Aug 24 04:42:37 2022] GET /identity/v3/services => generated 1735 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:42:37.619829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb18504-1bec-450a-988b-4814832b2bb5 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:37.691289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:37.736034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb18504-1bec-450a-988b-4814832b2bb5 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:37.774845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13c74c-899f-4213-b4d5-55d02e35c366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:37.776183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13c74c-899f-4213-b4d5-55d02e35c366 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:37.776448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13c74c-899f-4213-b4d5-55d02e35c366 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:37.776721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13c74c-899f-4213-b4d5-55d02e35c366 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:37.777303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13c74c-899f-4213-b4d5-55d02e35c366 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:37.778660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13c74c-899f-4213-b4d5-55d02e35c366 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-be13c74c-899f-4213-b4d5-55d02e35c366 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:37.787745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:37.788773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:37.789637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 500/999] 10.222.0.105 () {56 vars in 1131 bytes} [Wed Aug 24 04:42:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:37.797524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315dd796-2522-4291-b68d-2fe132646ef1 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:37.898739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315dd796-2522-4291-b68d-2fe132646ef1 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:37.933958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe9dbd-4b6b-4fd4-9960-403d4adadef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:37.934565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe9dbd-4b6b-4fd4-9960-403d4adadef7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:37.934843 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe9dbd-4b6b-4fd4-9960-403d4adadef7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:37.935151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe9dbd-4b6b-4fd4-9960-403d4adadef7 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:37.935776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe9dbd-4b6b-4fd4-9960-403d4adadef7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:37.936077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe9dbd-4b6b-4fd4-9960-403d4adadef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:37.937435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe9dbd-4b6b-4fd4-9960-403d4adadef7 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:37.942588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 500/1000] 10.222.0.105 () {56 vars in 1127 bytes} [Wed Aug 24 04:42:37 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:37.948302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13c74c-899f-4213-b4d5-55d02e35c366 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:38.051547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13c74c-899f-4213-b4d5-55d02e35c366 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:38.078467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:38.088209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bb6c7-4c51-4fcb-86eb-a8a1b0515426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:38.089919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb6c7-4c51-4fcb-86eb-a8a1b0515426 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:38.089919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb6c7-4c51-4fcb-86eb-a8a1b0515426 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:38.089919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb6c7-4c51-4fcb-86eb-a8a1b0515426 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:38.090157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bb6c7-4c51-4fcb-86eb-a8a1b0515426 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:38.090351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bb6c7-4c51-4fcb-86eb-a8a1b0515426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:38.091742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bb6c7-4c51-4fcb-86eb-a8a1b0515426 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:38.100868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 501/1001] 10.222.0.105 () {56 vars in 1127 bytes} [Wed Aug 24 04:42:37 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:38.102320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe9dbd-4b6b-4fd4-9960-403d4adadef7 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:38.146817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:38.213943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe9dbd-4b6b-4fd4-9960-403d4adadef7 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:38.216668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8977d-b2e3-4214-b739-29a81531b40a None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:38.217137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8977d-b2e3-4214-b739-29a81531b40a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:38.217243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8977d-b2e3-4214-b739-29a81531b40a None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:38.218674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 502/1002] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:38.223564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438a827-0016-427f-a186-891c9b90177c None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:38.223853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438a827-0016-427f-a186-891c9b90177c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:38.224130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9438a827-0016-427f-a186-891c9b90177c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:38.242017 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-9438a827-0016-427f-a186-891c9b90177c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:38.243231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9438a827-0016-427f-a186-891c9b90177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:38.252169 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a9e6-a55c-4c47-9851-bdf78cb5dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:38.252784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a9e6-a55c-4c47-9851-bdf78cb5dd98 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:38.253068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a9e6-a55c-4c47-9851-bdf78cb5dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:38.253342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a9e6-a55c-4c47-9851-bdf78cb5dd98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:38.254004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973a9e6-a55c-4c47-9851-bdf78cb5dd98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:38.255391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973a9e6-a55c-4c47-9851-bdf78cb5dd98 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:38.312058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 503/1003] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:38 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:38.327484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 501/1004] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:42:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:38.584231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438a827-0016-427f-a186-891c9b90177c None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:38.688597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9438a827-0016-427f-a186-891c9b90177c None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:38.710460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:38.733135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00436300-6827-465c-8e33-00e87f73ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:38.733772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00436300-6827-465c-8e33-00e87f73ee63 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:38.734048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00436300-6827-465c-8e33-00e87f73ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:38.734317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00436300-6827-465c-8e33-00e87f73ee63 None admin] PATH_INFO: `/v3/services/image` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:38.734894 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00436300-6827-465c-8e33-00e87f73ee63 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:38.736313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00436300-6827-465c-8e33-00e87f73ee63 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-00436300-6827-465c-8e33-00e87f73ee63 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:38.745124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:38.746119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:38.747064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 504/1005] 10.222.0.105 () {56 vars in 1110 bytes} [Wed Aug 24 04:42:38 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:38.754035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a9e6-a55c-4c47-9851-bdf78cb5dd98 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:38.854757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:38.858720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a9e6-a55c-4c47-9851-bdf78cb5dd98 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:38.893216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b78d18-857a-45d1-a7be-9b24e66f44b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:38.893815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78d18-857a-45d1-a7be-9b24e66f44b4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:38.894095 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78d18-857a-45d1-a7be-9b24e66f44b4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:38.894364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b78d18-857a-45d1-a7be-9b24e66f44b4 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:38.894980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b78d18-857a-45d1-a7be-9b24e66f44b4 None admin] RBAC: Adding query filter params (name=image) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:38.895310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b78d18-857a-45d1-a7be-9b24e66f44b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:38.896687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b78d18-857a-45d1-a7be-9b24e66f44b4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:38.906449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 502/1006] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:38 2022] GET /identity/v3/services?name=image => generated 123 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:38.907543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00436300-6827-465c-8e33-00e87f73ee63 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:39.010514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00436300-6827-465c-8e33-00e87f73ee63 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:39.044405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91ddd5-4859-4722-ae5b-76c6a080a99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:39.045009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ddd5-4859-4722-ae5b-76c6a080a99d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:39.045287 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ddd5-4859-4722-ae5b-76c6a080a99d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:39.045556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ddd5-4859-4722-ae5b-76c6a080a99d None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:39.046181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91ddd5-4859-4722-ae5b-76c6a080a99d None admin] RBAC: Adding query filter params (type=image) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:39.046482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91ddd5-4859-4722-ae5b-76c6a080a99d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:39.047927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91ddd5-4859-4722-ae5b-76c6a080a99d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:39.059125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 505/1007] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:38 2022] GET /identity/v3/services?type=image => generated 123 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:39.564218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517d4bd-5f17-4664-b14d-8cfa49493db1 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:39.564978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517d4bd-5f17-4664-b14d-8cfa49493db1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:39.565057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517d4bd-5f17-4664-b14d-8cfa49493db1 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:39.566501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 503/1008] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:39.571403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a59d6-2241-4d52-a82e-1595ef0212b3 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:39.571868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a59d6-2241-4d52-a82e-1595ef0212b3 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:39.572053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a59d6-2241-4d52-a82e-1595ef0212b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:39.589854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-cd5a59d6-2241-4d52-a82e-1595ef0212b3 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:39.591054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cd5a59d6-2241-4d52-a82e-1595ef0212b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:39.632937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:39.658575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 506/1009] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:39 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:39.929094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517d4bd-5f17-4664-b14d-8cfa49493db1 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:40.032787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517d4bd-5f17-4664-b14d-8cfa49493db1 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:40.068141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61012fe4-5165-4076-a550-600db2e50f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:40.068771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61012fe4-5165-4076-a550-600db2e50f0f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:40.069046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61012fe4-5165-4076-a550-600db2e50f0f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:40.069315 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61012fe4-5165-4076-a550-600db2e50f0f None admin] PATH_INFO: `/v3/services/compute` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:40.069890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61012fe4-5165-4076-a550-600db2e50f0f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:40.071266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61012fe4-5165-4076-a550-600db2e50f0f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-61012fe4-5165-4076-a550-600db2e50f0f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:40.080327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:40.081347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:40.082230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 504/1010] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:40.087202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a59d6-2241-4d52-a82e-1595ef0212b3 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:40.202227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a59d6-2241-4d52-a82e-1595ef0212b3 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:40.236623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882614cb-1848-4767-9c38-c9db4b28d252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:40.237239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882614cb-1848-4767-9c38-c9db4b28d252 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:40.237521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882614cb-1848-4767-9c38-c9db4b28d252 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:40.237794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882614cb-1848-4767-9c38-c9db4b28d252 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:40.238417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882614cb-1848-4767-9c38-c9db4b28d252 None admin] RBAC: Adding query filter params (name=compute) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:40.238716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882614cb-1848-4767-9c38-c9db4b28d252 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:40.240101 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882614cb-1848-4767-9c38-c9db4b28d252 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:40.244241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573c085-adb5-41f9-931b-b79ed4b003f0 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:40.244522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573c085-adb5-41f9-931b-b79ed4b003f0 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:40.249403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b573c085-adb5-41f9-931b-b79ed4b003f0 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:40.249528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 507/1011] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:40 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:40.249622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 505/1012] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:40.250853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882614cb-1848-4767-9c38-c9db4b28d252 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:40.251284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b7a4-e735-4450-b0ee-939387aec29e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:40.251568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b7a4-e735-4450-b0ee-939387aec29e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:40.251842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09b7a4-e735-4450-b0ee-939387aec29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:40.269601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-1b09b7a4-e735-4450-b0ee-939387aec29e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:40.270776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-1b09b7a4-e735-4450-b0ee-939387aec29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:40.295820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:40.298082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:40.342817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 506/1013] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:40 2022] POST /identity/v3/auth/tokens => generated 1720 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:40.358519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882614cb-1848-4767-9c38-c9db4b28d252 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:40.393776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50fa40-53b0-415f-b4c2-ce30de181ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:40.394383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50fa40-53b0-415f-b4c2-ce30de181ad9 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:40.394658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50fa40-53b0-415f-b4c2-ce30de181ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:40.394926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50fa40-53b0-415f-b4c2-ce30de181ad9 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:40.395662 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50fa40-53b0-415f-b4c2-ce30de181ad9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:40.395966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50fa40-53b0-415f-b4c2-ce30de181ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:40.397367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50fa40-53b0-415f-b4c2-ce30de181ad9 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:40.402968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 508/1014] 10.222.0.105 () {56 vars in 1117 bytes} [Wed Aug 24 04:42:40 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:40.408338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b7a4-e735-4450-b0ee-939387aec29e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:40.507594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09b7a4-e735-4450-b0ee-939387aec29e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:40.541349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afa148-b8f0-41c4-8ef3-6bd7683ec274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:40.541955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afa148-b8f0-41c4-8ef3-6bd7683ec274 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:40.542229 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afa148-b8f0-41c4-8ef3-6bd7683ec274 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:40.542496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afa148-b8f0-41c4-8ef3-6bd7683ec274 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:40.543179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afa148-b8f0-41c4-8ef3-6bd7683ec274 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:40.544522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3afa148-b8f0-41c4-8ef3-6bd7683ec274 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:40.615908 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50fa40-53b0-415f-b4c2-ce30de181ad9 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:40.623406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 507/1015] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:42:40 2022] POST /identity/v3/endpoints => generated 336 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:40.728015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50fa40-53b0-415f-b4c2-ce30de181ad9 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:40.730188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:40.771623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6629f1-902f-4afa-b7ab-2805d538e806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:40.772178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6629f1-902f-4afa-b7ab-2805d538e806 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:40.772457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6629f1-902f-4afa-b7ab-2805d538e806 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:40.772724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6629f1-902f-4afa-b7ab-2805d538e806 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:40.773939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6629f1-902f-4afa-b7ab-2805d538e806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:40.775977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6629f1-902f-4afa-b7ab-2805d538e806 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:40.779053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 509/1016] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:40 2022] GET /identity/v3/domains/default => generated 200 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:40.784144 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afa148-b8f0-41c4-8ef3-6bd7683ec274 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:40.803187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:40.893568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3afa148-b8f0-41c4-8ef3-6bd7683ec274 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:40.929983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d680ed4-9b4d-4921-ad55-e108b97d27b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:40.930593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d680ed4-9b4d-4921-ad55-e108b97d27b6 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:40.930873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d680ed4-9b4d-4921-ad55-e108b97d27b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:40.931246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d680ed4-9b4d-4921-ad55-e108b97d27b6 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:40.932195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d680ed4-9b4d-4921-ad55-e108b97d27b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:40.933624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d680ed4-9b4d-4921-ad55-e108b97d27b6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:40.998102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf026-a95d-42b1-a997-0f497de702c7 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:40.998390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf026-a95d-42b1-a997-0f497de702c7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:40.998717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbf026-a95d-42b1-a997-0f497de702c7 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:41.000232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 510/1017] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:41.005083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5efbed-e649-4437-b418-3b7f4012d6a1 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:41.005376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5efbed-e649-4437-b418-3b7f4012d6a1 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:41.005652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5efbed-e649-4437-b418-3b7f4012d6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:41.015993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 508/1018] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:42:40 2022] POST /identity/v3/projects => generated 312 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:41.030227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ae5efbed-e649-4437-b418-3b7f4012d6a1 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:41.031403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ae5efbed-e649-4437-b418-3b7f4012d6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:41.107131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 511/1019] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:41 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:41.387120 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d680ed4-9b4d-4921-ad55-e108b97d27b6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:41.455849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:41.492606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d680ed4-9b4d-4921-ad55-e108b97d27b6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:41.526634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26863005-2881-4e08-9448-6daea58bd5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:41.527270 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26863005-2881-4e08-9448-6daea58bd5e8 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:41.527548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26863005-2881-4e08-9448-6daea58bd5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:41.527816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26863005-2881-4e08-9448-6daea58bd5e8 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:41.528453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26863005-2881-4e08-9448-6daea58bd5e8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:41.530156 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26863005-2881-4e08-9448-6daea58bd5e8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:41.628654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 509/1020] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:42:41 2022] POST /identity/v3/services => generated 243 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:42:42.728809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2fb93-8f3e-453e-a155-bd2e7d3cde70 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:42.729336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2fb93-8f3e-453e-a155-bd2e7d3cde70 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:42.729416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2fb93-8f3e-453e-a155-bd2e7d3cde70 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:42.730810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 512/1021] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:42.736198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1ec1d-6d74-44de-bd80-abb141526259 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:42.736577 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1ec1d-6d74-44de-bd80-abb141526259 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:42.736780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1ec1d-6d74-44de-bd80-abb141526259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:42.755706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c6a1ec1d-6d74-44de-bd80-abb141526259 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:42.756889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c6a1ec1d-6d74-44de-bd80-abb141526259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:42.829027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 510/1022] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:42 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:43.000624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef286f6-3c1b-453c-912e-879030ba6e6c None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.000817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef286f6-3c1b-453c-912e-879030ba6e6c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.001094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef286f6-3c1b-453c-912e-879030ba6e6c None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.002554 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 513/1023] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:43.007610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e78c4-1258-48b1-9cd1-87a78b16e75d None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.007901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e78c4-1258-48b1-9cd1-87a78b16e75d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.008174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e78c4-1258-48b1-9cd1-87a78b16e75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.026641 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c26e78c4-1258-48b1-9cd1-87a78b16e75d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:43.027849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c26e78c4-1258-48b1-9cd1-87a78b16e75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:43.075432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:43.095376 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 511/1024] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:43 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:43.100955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef286f6-3c1b-453c-912e-879030ba6e6c None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:43.203867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:43.211138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef286f6-3c1b-453c-912e-879030ba6e6c None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:43.245769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff4a88-4bf8-46ab-ba73-91d97d0f27ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:43.246400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff4a88-4bf8-46ab-ba73-91d97d0f27ce None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.246681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff4a88-4bf8-46ab-ba73-91d97d0f27ce None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.246954 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff4a88-4bf8-46ab-ba73-91d97d0f27ce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.257346 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ff4a88-4bf8-46ab-ba73-91d97d0f27ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-15ff4a88-4bf8-46ab-ba73-91d97d0f27ce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:43.262747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:43.263837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:43.264671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 514/1025] 10.222.0.105 () {56 vars in 1125 bytes} [Wed Aug 24 04:42:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:43.269889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e78c4-1258-48b1-9cd1-87a78b16e75d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:43.367393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff4a88-4bf8-46ab-ba73-91d97d0f27ce None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:43.373350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e78c4-1258-48b1-9cd1-87a78b16e75d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:43.406968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e9559-5e54-4958-8a6d-ffbfc0aea16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:43.407623 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e9559-5e54-4958-8a6d-ffbfc0aea16b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.407900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e9559-5e54-4958-8a6d-ffbfc0aea16b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.408167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e9559-5e54-4958-8a6d-ffbfc0aea16b None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.408851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e9559-5e54-4958-8a6d-ffbfc0aea16b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:43.409147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e9559-5e54-4958-8a6d-ffbfc0aea16b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:43.410495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e9559-5e54-4958-8a6d-ffbfc0aea16b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:43.423199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 512/1026] 10.222.0.105 () {56 vars in 1120 bytes} [Wed Aug 24 04:42:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:43.425198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e9559-5e54-4958-8a6d-ffbfc0aea16b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:43.471960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:43.479452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ff4a88-4bf8-46ab-ba73-91d97d0f27ce None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:43.516253 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff0c38-a08c-4a77-b30f-3d19848673ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:43.516805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff0c38-a08c-4a77-b30f-3d19848673ff None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.517087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff0c38-a08c-4a77-b30f-3d19848673ff None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.517375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff0c38-a08c-4a77-b30f-3d19848673ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.518481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff0c38-a08c-4a77-b30f-3d19848673ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:43.520401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff0c38-a08c-4a77-b30f-3d19848673ff None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:43.522137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 515/1027] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:43 2022] GET /identity/v3/domains/default => generated 200 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:43.527670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff0c38-a08c-4a77-b30f-3d19848673ff None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:43.532535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e9559-5e54-4958-8a6d-ffbfc0aea16b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:43.554199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:43.569900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71881ff5-596b-4089-890d-373d553f4609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:43.570535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71881ff5-596b-4089-890d-373d553f4609 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.570810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71881ff5-596b-4089-890d-373d553f4609 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.571104 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71881ff5-596b-4089-890d-373d553f4609 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.582650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71881ff5-596b-4089-890d-373d553f4609 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:43.584117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71881ff5-596b-4089-890d-373d553f4609 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-71881ff5-596b-4089-890d-373d553f4609 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:43.598359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:43.599524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:43.600671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:43.600671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:43.600671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:43.605132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 513/1028] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:43.606409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e736144-c676-4fc2-8ccd-ae20e51abc7a None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.606695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e736144-c676-4fc2-8ccd-ae20e51abc7a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.606969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e736144-c676-4fc2-8ccd-ae20e51abc7a None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.608377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 514/1029] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:43.608987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e736144-c676-4fc2-8ccd-ae20e51abc7a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:43.636221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff0c38-a08c-4a77-b30f-3d19848673ff None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:43.670894 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af059dc1-eb9e-4e18-9ef9-7e3bdfdc007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:43.671519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af059dc1-eb9e-4e18-9ef9-7e3bdfdc007b None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.671795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af059dc1-eb9e-4e18-9ef9-7e3bdfdc007b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.672071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af059dc1-eb9e-4e18-9ef9-7e3bdfdc007b None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.672694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af059dc1-eb9e-4e18-9ef9-7e3bdfdc007b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:43.674136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af059dc1-eb9e-4e18-9ef9-7e3bdfdc007b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:43.703902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:43.710516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e736144-c676-4fc2-8ccd-ae20e51abc7a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:43.744431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb134e9-1638-4ff9-8798-de9f06509f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:43.745036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb134e9-1638-4ff9-8798-de9f06509f5d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.745288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb134e9-1638-4ff9-8798-de9f06509f5d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.745552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb134e9-1638-4ff9-8798-de9f06509f5d None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.746176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb134e9-1638-4ff9-8798-de9f06509f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:43.746470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb134e9-1638-4ff9-8798-de9f06509f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:43.747837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb134e9-1638-4ff9-8798-de9f06509f5d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:43.758519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 515/1030] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:43.759946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3dfed-ce03-4c99-94cc-e630d89f50bf None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.760224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3dfed-ce03-4c99-94cc-e630d89f50bf None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.760492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3dfed-ce03-4c99-94cc-e630d89f50bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.778476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-cdf3dfed-ce03-4c99-94cc-e630d89f50bf None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:43.779659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-cdf3dfed-ce03-4c99-94cc-e630d89f50bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:43.783905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 516/1031] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:42:43 2022] POST /identity/v3/users => generated 286 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:42:43.784654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af059dc1-eb9e-4e18-9ef9-7e3bdfdc007b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:43.855130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 516/1032] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:43 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:43.883693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:43.904628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af059dc1-eb9e-4e18-9ef9-7e3bdfdc007b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:43.946018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a96865-0cd4-4fac-9938-47a53f390410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:43.946018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96865-0cd4-4fac-9938-47a53f390410 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:43.946018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96865-0cd4-4fac-9938-47a53f390410 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:43.946018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a96865-0cd4-4fac-9938-47a53f390410 None admin] PATH_INFO: `/v3/users/nova` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:43.953406 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a96865-0cd4-4fac-9938-47a53f390410 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:43.955156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a96865-0cd4-4fac-9938-47a53f390410 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-c0a96865-0cd4-4fac-9938-47a53f390410 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:43.968107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:43.969092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:43.970089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 517/1033] 10.222.0.105 () {56 vars in 1098 bytes} [Wed Aug 24 04:42:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:43.975133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3dfed-ce03-4c99-94cc-e630d89f50bf None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:44.061939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:44.077489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf3dfed-ce03-4c99-94cc-e630d89f50bf None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:44.111818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd22d49-5b16-4101-bf95-4ef6bdcc7850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:44.112473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd22d49-5b16-4101-bf95-4ef6bdcc7850 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:44.112746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd22d49-5b16-4101-bf95-4ef6bdcc7850 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:44.113011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd22d49-5b16-4101-bf95-4ef6bdcc7850 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:44.113769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd22d49-5b16-4101-bf95-4ef6bdcc7850 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:44.114070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd22d49-5b16-4101-bf95-4ef6bdcc7850 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:44.115511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd22d49-5b16-4101-bf95-4ef6bdcc7850 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:44.127909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a96865-0cd4-4fac-9938-47a53f390410 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:44.139877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 517/1034] 10.222.0.105 () {56 vars in 1138 bytes} [Wed Aug 24 04:42:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:44.141205 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd22d49-5b16-4101-bf95-4ef6bdcc7850 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:44.247935 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a96865-0cd4-4fac-9938-47a53f390410 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:44.250159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd22d49-5b16-4101-bf95-4ef6bdcc7850 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:44.285170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb90f9e-e557-4d37-acc4-0f7623809bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:44.286063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb90f9e-e557-4d37-acc4-0f7623809bf9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:44.286341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb90f9e-e557-4d37-acc4-0f7623809bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:44.286609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb90f9e-e557-4d37-acc4-0f7623809bf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:44.288659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461124e7-79f7-4361-aa20-1d8db9a5b765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:44.289393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461124e7-79f7-4361-aa20-1d8db9a5b765 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:44.289666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461124e7-79f7-4361-aa20-1d8db9a5b765 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:44.290662 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461124e7-79f7-4361-aa20-1d8db9a5b765 None admin] PATH_INFO: `/v3/services/image` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:44.291109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461124e7-79f7-4361-aa20-1d8db9a5b765 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:44.292471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461124e7-79f7-4361-aa20-1d8db9a5b765 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:44.301401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-461124e7-79f7-4361-aa20-1d8db9a5b765 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:44.301556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:44.302513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 04:42:44.303426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:44.307350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 518/1035] 10.222.0.105 () {56 vars in 1110 bytes} [Wed Aug 24 04:42:44 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:44.309265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461124e7-79f7-4361-aa20-1d8db9a5b765 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:44.316187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:44.360798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb90f9e-e557-4d37-acc4-0f7623809bf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:44.362935 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb90f9e-e557-4d37-acc4-0f7623809bf9 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:44.417754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461124e7-79f7-4361-aa20-1d8db9a5b765 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:44.453914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970e587-e5ee-4196-836d-caf19585197a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:44.454781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970e587-e5ee-4196-836d-caf19585197a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:44.455249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970e587-e5ee-4196-836d-caf19585197a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:44.455669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970e587-e5ee-4196-836d-caf19585197a None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:44.456444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970e587-e5ee-4196-836d-caf19585197a None admin] RBAC: Adding query filter params (name=image) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:44.456892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970e587-e5ee-4196-836d-caf19585197a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:44.458251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970e587-e5ee-4196-836d-caf19585197a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:44.468907 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 519/1036] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:44 2022] GET /identity/v3/services?name=image => generated 123 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:44.469647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970e587-e5ee-4196-836d-caf19585197a None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:44.474600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 518/1037] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:44 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 04:42:44.475857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb90f9e-e557-4d37-acc4-0f7623809bf9 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:44.522301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:44.547209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:44.579929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb90f9e-e557-4d37-acc4-0f7623809bf9 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:44.582648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970e587-e5ee-4196-836d-caf19585197a None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:44.614371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd166d51-0529-4692-b849-6a70c8c4e8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:44.614993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd166d51-0529-4692-b849-6a70c8c4e8fc None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:44.615295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd166d51-0529-4692-b849-6a70c8c4e8fc None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:44.615563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd166d51-0529-4692-b849-6a70c8c4e8fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:44.617239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb8f2a-d7dc-4f74-bc8d-4b24efc9b11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:44.617836 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb8f2a-d7dc-4f74-bc8d-4b24efc9b11e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:44.618113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb8f2a-d7dc-4f74-bc8d-4b24efc9b11e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:44.618917 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb8f2a-d7dc-4f74-bc8d-4b24efc9b11e None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:44.619569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb8f2a-d7dc-4f74-bc8d-4b24efc9b11e None admin] RBAC: Adding query filter params (type=image) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:44.619870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb8f2a-d7dc-4f74-bc8d-4b24efc9b11e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:44.621222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb8f2a-d7dc-4f74-bc8d-4b24efc9b11e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:44.628128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd166d51-0529-4692-b849-6a70c8c4e8fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:44.629544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd166d51-0529-4692-b849-6a70c8c4e8fc None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:44.631340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 520/1038] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:44 2022] GET /identity/v3/services?type=image => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:44.632190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb8f2a-d7dc-4f74-bc8d-4b24efc9b11e None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-fd166d51-0529-4692-b849-6a70c8c4e8fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:44.646398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:44.647618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:44.648557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:44.648557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:44.648557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:44.648557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 519/1039] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:44.653870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd166d51-0529-4692-b849-6a70c8c4e8fc None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:44.743711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb8f2a-d7dc-4f74-bc8d-4b24efc9b11e None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:44.772618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:44.774896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:44.779946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b540279-e7ef-4b2c-a523-b2874fc86602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:44.780093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd166d51-0529-4692-b849-6a70c8c4e8fc None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:44.780624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b540279-e7ef-4b2c-a523-b2874fc86602 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:44.780898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b540279-e7ef-4b2c-a523-b2874fc86602 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:44.781164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b540279-e7ef-4b2c-a523-b2874fc86602 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:44.781857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b540279-e7ef-4b2c-a523-b2874fc86602 None admin] RBAC: Adding query filter params (interface=public, service_id=81a4e673b6c34924aecc224a4c74a97a, region_id=RegionOne) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:44.782838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b540279-e7ef-4b2c-a523-b2874fc86602 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:44.783526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b540279-e7ef-4b2c-a523-b2874fc86602 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:44.794304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 521/1040] 10.222.0.105 () {56 vars in 1256 bytes} [Wed Aug 24 04:42:44 2022] GET /identity/v3/endpoints?service_id=81a4e673b6c34924aecc224a4c74a97a&interface=public®ion_id=RegionOne => generated 195 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:44.795478 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b540279-e7ef-4b2c-a523-b2874fc86602 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:44.815816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aa54c-f8ca-473d-b0a3-a4866db64522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:44.816418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aa54c-f8ca-473d-b0a3-a4866db64522 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:44.816686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aa54c-f8ca-473d-b0a3-a4866db64522 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:44.816939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508aa54c-f8ca-473d-b0a3-a4866db64522 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:44.820882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aa54c-f8ca-473d-b0a3-a4866db64522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:44.820882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aa54c-f8ca-473d-b0a3-a4866db64522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:44.820882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508aa54c-f8ca-473d-b0a3-a4866db64522 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:44.830850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 520/1041] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:44.836634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aa54c-f8ca-473d-b0a3-a4866db64522 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:44.911153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b540279-e7ef-4b2c-a523-b2874fc86602 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:44.945575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508aa54c-f8ca-473d-b0a3-a4866db64522 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:44.952982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce924b-0329-4f9d-b50d-d96bdf585d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:44.953592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce924b-0329-4f9d-b50d-d96bdf585d7a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:44.953867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce924b-0329-4f9d-b50d-d96bdf585d7a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:44.954132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce924b-0329-4f9d-b50d-d96bdf585d7a None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:44.954735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce924b-0329-4f9d-b50d-d96bdf585d7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:44.956106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce924b-0329-4f9d-b50d-d96bdf585d7a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:44.963040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 522/1042] 10.222.0.105 () {56 vars in 1092 bytes} [Wed Aug 24 04:42:44 2022] GET /identity/v3/services => generated 1966 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:42:44.996018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3061b-cd2c-4f35-bf59-ae8ca64ff4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:44.996673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3061b-cd2c-4f35-bf59-ae8ca64ff4c1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:44.996952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3061b-cd2c-4f35-bf59-ae8ca64ff4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:44.997222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3061b-cd2c-4f35-bf59-ae8ca64ff4c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:45.004903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3061b-cd2c-4f35-bf59-ae8ca64ff4c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:45.006395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c3061b-cd2c-4f35-bf59-ae8ca64ff4c1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-08c3061b-cd2c-4f35-bf59-ae8ca64ff4c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:45.019153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:45.020150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:45.021090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:45.021090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:45.021090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:45.021090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:45.021090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:45.021090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:45.021090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:45.021090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:45.023380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 521/1043] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:45.026714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce924b-0329-4f9d-b50d-d96bdf585d7a None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:45.093716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:45.139411 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce924b-0329-4f9d-b50d-d96bdf585d7a None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:45.176919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ae928-cff1-4a76-a1b5-6ebf3d426f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:45.177534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ae928-cff1-4a76-a1b5-6ebf3d426f91 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:45.177814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ae928-cff1-4a76-a1b5-6ebf3d426f91 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:45.178084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ae928-cff1-4a76-a1b5-6ebf3d426f91 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:45.178774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ae928-cff1-4a76-a1b5-6ebf3d426f91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:45.179109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ae928-cff1-4a76-a1b5-6ebf3d426f91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:45.180526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ae928-cff1-4a76-a1b5-6ebf3d426f91 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:45.193617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 523/1044] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:45.199586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3061b-cd2c-4f35-bf59-ae8ca64ff4c1 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:45.219542 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:45.301813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c3061b-cd2c-4f35-bf59-ae8ca64ff4c1 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:45.337556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb332f4-78dd-48c3-ad51-f74d32f6e12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:45.338257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb332f4-78dd-48c3-ad51-f74d32f6e12c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:45.338537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb332f4-78dd-48c3-ad51-f74d32f6e12c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:45.338806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb332f4-78dd-48c3-ad51-f74d32f6e12c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:45.339678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb332f4-78dd-48c3-ad51-f74d32f6e12c None admin] RBAC: Adding query filter params (role.id=dced619d94db43ed8cccb9c80be594b9, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=3c822c8f23b74bae84769046fdb85db4) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:45.339985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb332f4-78dd-48c3-ad51-f74d32f6e12c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:45.341391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb332f4-78dd-48c3-ad51-f74d32f6e12c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:45.351127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 522/1045] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:45 2022] GET /identity/v3/role_assignments?user.id=3c822c8f23b74bae84769046fdb85db4&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=dced619d94db43ed8cccb9c80be594b9 => generated 260 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:45.775317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c6459-9565-464f-8d97-66855f335883 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:45.775677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c6459-9565-464f-8d97-66855f335883 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:45.775950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c6459-9565-464f-8d97-66855f335883 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:45.777306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 524/1046] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:45.782571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d8012-91c6-401e-a741-e8367bea59cc None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:45.783010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d8012-91c6-401e-a741-e8367bea59cc None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:45.783210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d8012-91c6-401e-a741-e8367bea59cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:45.800950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9b9d8012-91c6-401e-a741-e8367bea59cc None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:45.802132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9b9d8012-91c6-401e-a741-e8367bea59cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:45.871177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 523/1047] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:45 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:46.137107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c6459-9565-464f-8d97-66855f335883 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:46.245784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c6459-9565-464f-8d97-66855f335883 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:46.266657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:46.280908 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26190bfb-dfc0-41f7-b21c-d97a62e918f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:46.281547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26190bfb-dfc0-41f7-b21c-d97a62e918f4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:46.281829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26190bfb-dfc0-41f7-b21c-d97a62e918f4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:46.282102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26190bfb-dfc0-41f7-b21c-d97a62e918f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:46.293169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26190bfb-dfc0-41f7-b21c-d97a62e918f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-26190bfb-dfc0-41f7-b21c-d97a62e918f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:46.298147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:46.299399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:46.300481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:46.308867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d8012-91c6-401e-a741-e8367bea59cc None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:46.310947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 525/1048] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:46.321105 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:46.412355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d8012-91c6-401e-a741-e8367bea59cc None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:46.448131 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2639d1-edc5-4a3a-ae60-49d01ef772a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:46.448735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2639d1-edc5-4a3a-ae60-49d01ef772a6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:46.449010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2639d1-edc5-4a3a-ae60-49d01ef772a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:46.449273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2639d1-edc5-4a3a-ae60-49d01ef772a6 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:46.449914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2639d1-edc5-4a3a-ae60-49d01ef772a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:46.450206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2639d1-edc5-4a3a-ae60-49d01ef772a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:46.451581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2639d1-edc5-4a3a-ae60-49d01ef772a6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:46.459862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 524/1049] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:42:46 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:46.465642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26190bfb-dfc0-41f7-b21c-d97a62e918f4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:46.566741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26190bfb-dfc0-41f7-b21c-d97a62e918f4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:46.600215 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc16b19-d3dc-4692-b8d5-5e7be1d2d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:46.600832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc16b19-d3dc-4692-b8d5-5e7be1d2d24f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:46.601105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc16b19-d3dc-4692-b8d5-5e7be1d2d24f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:46.601369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc16b19-d3dc-4692-b8d5-5e7be1d2d24f None admin] PATH_INFO: `/v3/users/778e0e2a2d0b4443a41b143cfd1e2d63` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:46.614954 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc16b19-d3dc-4692-b8d5-5e7be1d2d24f None admin] RBAC: Authorizing `identity:get_user(user_id=778e0e2a2d0b4443a41b143cfd1e2d63)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:46.616401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc16b19-d3dc-4692-b8d5-5e7be1d2d24f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:46.618750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 526/1050] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:42:46 2022] GET /identity/v3/users/778e0e2a2d0b4443a41b143cfd1e2d63 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:46.624525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2639d1-edc5-4a3a-ae60-49d01ef772a6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:46.695051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:46.725850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2639d1-edc5-4a3a-ae60-49d01ef772a6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:46.761528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431ac15-607d-4ecb-b139-0ba1628aa08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:46.762130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431ac15-607d-4ecb-b139-0ba1628aa08d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:46.762409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431ac15-607d-4ecb-b139-0ba1628aa08d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:46.762719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431ac15-607d-4ecb-b139-0ba1628aa08d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:46.830298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431ac15-607d-4ecb-b139-0ba1628aa08d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:46.831755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431ac15-607d-4ecb-b139-0ba1628aa08d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:46.876984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f687b-e407-4246-9fd3-51298cfd9a52 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:46.877316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f687b-e407-4246-9fd3-51298cfd9a52 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:46.877522 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f687b-e407-4246-9fd3-51298cfd9a52 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:46.879962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 527/1051] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:46.884268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68197a5e-da6b-42dd-ba56-16b79b01c9de None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:46.884554 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68197a5e-da6b-42dd-ba56-16b79b01c9de None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:46.884821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68197a5e-da6b-42dd-ba56-16b79b01c9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:46.902958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-68197a5e-da6b-42dd-ba56-16b79b01c9de None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:46.904856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-68197a5e-da6b-42dd-ba56-16b79b01c9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:46.905308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:46.936100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:46.937881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 525/1052] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:46 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:42:46.943715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431ac15-607d-4ecb-b139-0ba1628aa08d None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:46.977368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 528/1053] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:46 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:47.050491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431ac15-607d-4ecb-b139-0ba1628aa08d None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:47.085009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc73ee-9ff7-468b-ae4b-c508cedb506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:47.085641 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc73ee-9ff7-468b-ae4b-c508cedb506b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:47.085920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc73ee-9ff7-468b-ae4b-c508cedb506b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:47.086191 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc73ee-9ff7-468b-ae4b-c508cedb506b None admin] PATH_INFO: `/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:47.087938 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc73ee-9ff7-468b-ae4b-c508cedb506b None admin] RBAC: Authorizing `identity:get_project(project_id=eb211c3ed5824bbea2a5667dc1fb752a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:47.089355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc73ee-9ff7-468b-ae4b-c508cedb506b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:47.095913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 526/1054] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:42:46 2022] GET /identity/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:47.097153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68197a5e-da6b-42dd-ba56-16b79b01c9de None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:47.206919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68197a5e-da6b-42dd-ba56-16b79b01c9de None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:47.243772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736a1c7-232a-4b04-ae65-dd25527badad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:47.244314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736a1c7-232a-4b04-ae65-dd25527badad None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:47.244592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736a1c7-232a-4b04-ae65-dd25527badad None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:47.244862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736a1c7-232a-4b04-ae65-dd25527badad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:47.245587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736a1c7-232a-4b04-ae65-dd25527badad None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=eb211c3ed5824bbea2a5667dc1fb752a, user.id=778e0e2a2d0b4443a41b143cfd1e2d63) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:47.245952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736a1c7-232a-4b04-ae65-dd25527badad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:47.247587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736a1c7-232a-4b04-ae65-dd25527badad None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:47.248161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef849182-9915-4463-be9e-a7bedc241f94 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:47.248438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef849182-9915-4463-be9e-a7bedc241f94 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:47.248706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef849182-9915-4463-be9e-a7bedc241f94 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:47.250196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 527/1055] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:47.250725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef849182-9915-4463-be9e-a7bedc241f94 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:47.254074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 529/1056] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:47 2022] GET /identity/v3/role_assignments?user.id=778e0e2a2d0b4443a41b143cfd1e2d63&scope.project.id=eb211c3ed5824bbea2a5667dc1fb752a&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:47.255350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f288bf1-67eb-4f13-b27a-5d1b73c95dc0 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:47.255635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f288bf1-67eb-4f13-b27a-5d1b73c95dc0 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:47.255969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f288bf1-67eb-4f13-b27a-5d1b73c95dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:47.274356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-6f288bf1-67eb-4f13-b27a-5d1b73c95dc0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:47.275634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6f288bf1-67eb-4f13-b27a-5d1b73c95dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:47.304682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:47.307160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:47.356175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 530/1057] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:47 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:47.377278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef849182-9915-4463-be9e-a7bedc241f94 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:47.419581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd0545-c5f7-48b9-aa64-c7da714b8dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:47.420214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd0545-c5f7-48b9-aa64-c7da714b8dad None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:47.420498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd0545-c5f7-48b9-aa64-c7da714b8dad None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:47.420769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd0545-c5f7-48b9-aa64-c7da714b8dad None admin] PATH_INFO: `/v3/services/image` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:47.421349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd0545-c5f7-48b9-aa64-c7da714b8dad None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:47.422709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd0545-c5f7-48b9-aa64-c7da714b8dad None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-74fd0545-c5f7-48b9-aa64-c7da714b8dad None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:47.431391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:47.432424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:47.433280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 528/1058] 10.222.0.105 () {56 vars in 1110 bytes} [Wed Aug 24 04:42:47 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:47.438327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f288bf1-67eb-4f13-b27a-5d1b73c95dc0 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:47.538625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f288bf1-67eb-4f13-b27a-5d1b73c95dc0 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:47.578702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cb950-30a1-43fb-8eac-62b29e324578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:47.579568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cb950-30a1-43fb-8eac-62b29e324578 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:47.579978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cb950-30a1-43fb-8eac-62b29e324578 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:47.580371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cb950-30a1-43fb-8eac-62b29e324578 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:47.581159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cb950-30a1-43fb-8eac-62b29e324578 None admin] RBAC: Adding query filter params (name=image) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:47.581456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cb950-30a1-43fb-8eac-62b29e324578 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:47.582809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cb950-30a1-43fb-8eac-62b29e324578 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:47.588402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 531/1059] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:47 2022] GET /identity/v3/services?name=image => generated 123 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:47.593875 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd0545-c5f7-48b9-aa64-c7da714b8dad None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:47.633829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cb950-30a1-43fb-8eac-62b29e324578 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:47.662694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:47.698863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd0545-c5f7-48b9-aa64-c7da714b8dad None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:47.701235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:47.735299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d25b1-b06d-4413-8f27-775efc7c9192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:47.735912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d25b1-b06d-4413-8f27-775efc7c9192 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:47.736194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d25b1-b06d-4413-8f27-775efc7c9192 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:47.736463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d25b1-b06d-4413-8f27-775efc7c9192 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:47.737087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d25b1-b06d-4413-8f27-775efc7c9192 None admin] RBAC: Adding query filter params (type=image) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:47.737386 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d25b1-b06d-4413-8f27-775efc7c9192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:47.738745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d25b1-b06d-4413-8f27-775efc7c9192 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:47.740378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cb950-30a1-43fb-8eac-62b29e324578 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:47.747050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 529/1060] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:47 2022] GET /identity/v3/services?type=image => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:47.749235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d25b1-b06d-4413-8f27-775efc7c9192 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:47.782659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b3b4a-f386-4e70-aa1e-e4dd2c370a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:47.783320 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b3b4a-f386-4e70-aa1e-e4dd2c370a6d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:47.783599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b3b4a-f386-4e70-aa1e-e4dd2c370a6d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:47.783868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b3b4a-f386-4e70-aa1e-e4dd2c370a6d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:47.795162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b3b4a-f386-4e70-aa1e-e4dd2c370a6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-1b5b3b4a-f386-4e70-aa1e-e4dd2c370a6d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:47.799807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:47.800772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:47.801668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 532/1061] 10.222.0.105 () {56 vars in 1125 bytes} [Wed Aug 24 04:42:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:47.806811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b3b4a-f386-4e70-aa1e-e4dd2c370a6d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:47.857062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d25b1-b06d-4413-8f27-775efc7c9192 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:47.891483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef6c10-55fb-4bb1-9d6c-686b9a7e8aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:47.892094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef6c10-55fb-4bb1-9d6c-686b9a7e8aec None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:47.892376 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef6c10-55fb-4bb1-9d6c-686b9a7e8aec None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:47.892649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef6c10-55fb-4bb1-9d6c-686b9a7e8aec None admin] PATH_INFO: `/v3/endpoints` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:47.893310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef6c10-55fb-4bb1-9d6c-686b9a7e8aec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:47.894664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ef6c10-55fb-4bb1-9d6c-686b9a7e8aec None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:47.902504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:47.918025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b3b4a-f386-4e70-aa1e-e4dd2c370a6d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:47.952206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db69766-6493-4f60-92ae-f3e2643cf3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:47.952804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db69766-6493-4f60-92ae-f3e2643cf3f6 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:47.953076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db69766-6493-4f60-92ae-f3e2643cf3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:47.953387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db69766-6493-4f60-92ae-f3e2643cf3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:47.954037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db69766-6493-4f60-92ae-f3e2643cf3f6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:47.954332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db69766-6493-4f60-92ae-f3e2643cf3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:47.955694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db69766-6493-4f60-92ae-f3e2643cf3f6 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:47.964271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 533/1062] 10.222.0.105 () {56 vars in 1120 bytes} [Wed Aug 24 04:42:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:47.969869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db69766-6493-4f60-92ae-f3e2643cf3f6 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:47.977302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 530/1063] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:42:47 2022] POST /identity/v3/endpoints => generated 329 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 04:42:48.092142 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db69766-6493-4f60-92ae-f3e2643cf3f6 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:48.137313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def4f23-edf1-48c9-ae85-44ca9ea210e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:48.137949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def4f23-edf1-48c9-ae85-44ca9ea210e7 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:48.138227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def4f23-edf1-48c9-ae85-44ca9ea210e7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:48.138492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def4f23-edf1-48c9-ae85-44ca9ea210e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:48.150102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def4f23-edf1-48c9-ae85-44ca9ea210e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:48.151577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def4f23-edf1-48c9-ae85-44ca9ea210e7 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:48.154240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-8def4f23-edf1-48c9-ae85-44ca9ea210e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:48.166290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:48.167315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:48.168252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:48.168252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:48.168252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:48.168252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 534/1064] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:48.173765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef6c10-55fb-4bb1-9d6c-686b9a7e8aec None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:48.205748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:48.282527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef6c10-55fb-4bb1-9d6c-686b9a7e8aec None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:48.316534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2a5ae-3a96-4fd1-9627-1f17daf18d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:48.317145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a5ae-3a96-4fd1-9627-1f17daf18d60 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:48.317422 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a5ae-3a96-4fd1-9627-1f17daf18d60 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:48.317691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2a5ae-3a96-4fd1-9627-1f17daf18d60 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:48.318324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2a5ae-3a96-4fd1-9627-1f17daf18d60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:48.318620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2a5ae-3a96-4fd1-9627-1f17daf18d60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:48.320078 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2a5ae-3a96-4fd1-9627-1f17daf18d60 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:48.333556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 531/1065] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:48.340082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def4f23-edf1-48c9-ae85-44ca9ea210e7 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:48.441967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def4f23-edf1-48c9-ae85-44ca9ea210e7 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:48.475630 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd8a16-2c52-4b18-ae80-db890cf40f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:48.476257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd8a16-2c52-4b18-ae80-db890cf40f0c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:48.476534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd8a16-2c52-4b18-ae80-db890cf40f0c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:48.476799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd8a16-2c52-4b18-ae80-db890cf40f0c None admin] PATH_INFO: `/v3/users/nova` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:48.484929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd8a16-2c52-4b18-ae80-db890cf40f0c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:48.486360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bd8a16-2c52-4b18-ae80-db890cf40f0c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-d2bd8a16-2c52-4b18-ae80-db890cf40f0c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:48.497600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:48.498617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:48.499668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 535/1066] 10.222.0.105 () {56 vars in 1098 bytes} [Wed Aug 24 04:42:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:48.506224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2a5ae-3a96-4fd1-9627-1f17daf18d60 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:48.594896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:48.610433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2a5ae-3a96-4fd1-9627-1f17daf18d60 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:48.644716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22900b-696c-41fe-b67c-8216f635cf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:48.645317 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22900b-696c-41fe-b67c-8216f635cf18 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:48.645595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22900b-696c-41fe-b67c-8216f635cf18 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:48.645865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b22900b-696c-41fe-b67c-8216f635cf18 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:48.646580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22900b-696c-41fe-b67c-8216f635cf18 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:48.646883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22900b-696c-41fe-b67c-8216f635cf18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:48.648332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b22900b-696c-41fe-b67c-8216f635cf18 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:48.670784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 532/1067] 10.222.0.105 () {56 vars in 1138 bytes} [Wed Aug 24 04:42:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:48.672159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd8a16-2c52-4b18-ae80-db890cf40f0c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:48.732037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:48.779123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd8a16-2c52-4b18-ae80-db890cf40f0c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:48.813523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb7927-0e4a-4c4b-b965-bfdbded1d697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:48.814403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb7927-0e4a-4c4b-b965-bfdbded1d697 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:48.814685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb7927-0e4a-4c4b-b965-bfdbded1d697 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:48.814955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb7927-0e4a-4c4b-b965-bfdbded1d697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:48.825724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb7927-0e4a-4c4b-b965-bfdbded1d697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:48.827196 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb7927-0e4a-4c4b-b965-bfdbded1d697 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-4efb7927-0e4a-4c4b-b965-bfdbded1d697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:48.841316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:48.842298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:48.843222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:48.843222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:48.843222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:48.847365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 536/1068] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:48.848592 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22900b-696c-41fe-b67c-8216f635cf18 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:48.951187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b22900b-696c-41fe-b67c-8216f635cf18 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:48.986482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7f33d0-7718-41c1-baca-690379b15acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:48.987052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f33d0-7718-41c1-baca-690379b15acb None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:48.987316 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f33d0-7718-41c1-baca-690379b15acb None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:48.987589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f33d0-7718-41c1-baca-690379b15acb None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:48.988221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f33d0-7718-41c1-baca-690379b15acb None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:48.988522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f33d0-7718-41c1-baca-690379b15acb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:48.989868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7f33d0-7718-41c1-baca-690379b15acb None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:49.005607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 533/1069] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:49.007060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb7927-0e4a-4c4b-b965-bfdbded1d697 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:49.111828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb7927-0e4a-4c4b-b965-bfdbded1d697 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:49.120960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:49.145347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ab1ce-727e-4ffa-81a2-1960705df7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:49.146029 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ab1ce-727e-4ffa-81a2-1960705df7f7 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:49.146343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ab1ce-727e-4ffa-81a2-1960705df7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:49.146648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ab1ce-727e-4ffa-81a2-1960705df7f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:49.154692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ab1ce-727e-4ffa-81a2-1960705df7f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:49.156174 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ab1ce-727e-4ffa-81a2-1960705df7f7 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-620ab1ce-727e-4ffa-81a2-1960705df7f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:49.168023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:49.168909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:49.174737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:49.174737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:49.174737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:49.174737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:49.174737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:49.174737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:49.174737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:49.174737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:49.174737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 537/1070] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:49.175277 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25b4fa-879c-4ec8-ba97-e3d826305412 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:49.175277 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25b4fa-879c-4ec8-ba97-e3d826305412 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:49.175277 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25b4fa-879c-4ec8-ba97-e3d826305412 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:49.175277 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 534/1071] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:49.177485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ab1ce-727e-4ffa-81a2-1960705df7f7 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:49.178581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989b396-f45b-4eb3-b280-b3b1f9c4a845 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:49.178866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989b396-f45b-4eb3-b280-b3b1f9c4a845 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:49.179175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989b396-f45b-4eb3-b280-b3b1f9c4a845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:49.186403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:49.201677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-f989b396-f45b-4eb3-b280-b3b1f9c4a845 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:49.202854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f989b396-f45b-4eb3-b280-b3b1f9c4a845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:49.276416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 535/1072] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:49 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:49.285450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ab1ce-727e-4ffa-81a2-1960705df7f7 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:49.320319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005152a-8b4f-4f1f-9a9d-e0d9679e8b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:49.320917 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005152a-8b4f-4f1f-9a9d-e0d9679e8b00 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:49.321228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005152a-8b4f-4f1f-9a9d-e0d9679e8b00 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:49.321521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005152a-8b4f-4f1f-9a9d-e0d9679e8b00 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:49.322210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005152a-8b4f-4f1f-9a9d-e0d9679e8b00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:49.322514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005152a-8b4f-4f1f-9a9d-e0d9679e8b00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:49.323944 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005152a-8b4f-4f1f-9a9d-e0d9679e8b00 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:49.336265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 538/1073] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:49.342227 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989b396-f45b-4eb3-b280-b3b1f9c4a845 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:49.441439 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989b396-f45b-4eb3-b280-b3b1f9c4a845 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:49.470018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:49.475421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c72c4-61b7-4699-bcbe-fb26a90461f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:49.476084 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c72c4-61b7-4699-bcbe-fb26a90461f3 None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:49.476360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c72c4-61b7-4699-bcbe-fb26a90461f3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:49.476630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3c72c4-61b7-4699-bcbe-fb26a90461f3 None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/3c822c8f23b74bae84769046fdb85db4/roles/dced619d94db43ed8cccb9c80be594b9` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:49.479339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c72c4-61b7-4699-bcbe-fb26a90461f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=3c822c8f23b74bae84769046fdb85db4, role_id=dced619d94db43ed8cccb9c80be594b9)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:49.480757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3c72c4-61b7-4699-bcbe-fb26a90461f3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:49.549741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005152a-8b4f-4f1f-9a9d-e0d9679e8b00 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:49.578808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 536/1074] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:42:49 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/3c822c8f23b74bae84769046fdb85db4/roles/dced619d94db43ed8cccb9c80be594b9 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:42:49.580359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:49.660517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005152a-8b4f-4f1f-9a9d-e0d9679e8b00 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:49.697593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be22f3-ccdd-4cf2-bca6-be2884171dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:49.698237 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be22f3-ccdd-4cf2-bca6-be2884171dba None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:49.698518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be22f3-ccdd-4cf2-bca6-be2884171dba None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:49.698788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be22f3-ccdd-4cf2-bca6-be2884171dba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:49.710212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61be22f3-ccdd-4cf2-bca6-be2884171dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-61be22f3-ccdd-4cf2-bca6-be2884171dba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:49.715802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:49.716728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:49.718916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 539/1075] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:49.723679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c72c4-61b7-4699-bcbe-fb26a90461f3 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:49.824846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3c72c4-61b7-4699-bcbe-fb26a90461f3 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:49.859932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d871ad9-d105-4930-8763-2d26fbd05609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:49.860543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d871ad9-d105-4930-8763-2d26fbd05609 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:49.860822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d871ad9-d105-4930-8763-2d26fbd05609 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:49.861094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d871ad9-d105-4930-8763-2d26fbd05609 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:49.861754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d871ad9-d105-4930-8763-2d26fbd05609 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:49.862195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d871ad9-d105-4930-8763-2d26fbd05609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:49.863625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d871ad9-d105-4930-8763-2d26fbd05609 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:49.876610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 537/1076] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:42:49 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:49.878264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be22f3-ccdd-4cf2-bca6-be2884171dba None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:49.932685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6504dfe-9572-42de-aa70-2bda51acd1b1 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:49.932977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6504dfe-9572-42de-aa70-2bda51acd1b1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:49.933251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6504dfe-9572-42de-aa70-2bda51acd1b1 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:49.934806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 538/1077] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:49.939858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebad2b-370d-4731-b796-479ddae7c499 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:49.940137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebad2b-370d-4731-b796-479ddae7c499 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:49.940408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebad2b-370d-4731-b796-479ddae7c499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:49.958458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-cbebad2b-370d-4731-b796-479ddae7c499 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:49.959681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-cbebad2b-370d-4731-b796-479ddae7c499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:49.970019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:49.983945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be22f3-ccdd-4cf2-bca6-be2884171dba None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:50.019281 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:50.021509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472c6b9-c70f-424b-9876-9bb986f31124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:50.022136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c6b9-c70f-424b-9876-9bb986f31124 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:50.022418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c6b9-c70f-424b-9876-9bb986f31124 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:50.022692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472c6b9-c70f-424b-9876-9bb986f31124 None admin] PATH_INFO: `/v3/users/778e0e2a2d0b4443a41b143cfd1e2d63` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:50.023892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472c6b9-c70f-424b-9876-9bb986f31124 None admin] RBAC: Authorizing `identity:get_user(user_id=778e0e2a2d0b4443a41b143cfd1e2d63)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:50.026027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472c6b9-c70f-424b-9876-9bb986f31124 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:50.031059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 540/1078] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:42:49 2022] GET /identity/v3/users/778e0e2a2d0b4443a41b143cfd1e2d63 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:50.035108 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472c6b9-c70f-424b-9876-9bb986f31124 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:50.039235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 539/1079] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:49 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:50.140054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472c6b9-c70f-424b-9876-9bb986f31124 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:50.175059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552932c-8f6b-4ce4-afc3-ce61a513603d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:50.175059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552932c-8f6b-4ce4-afc3-ce61a513603d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:50.175059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552932c-8f6b-4ce4-afc3-ce61a513603d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:50.175059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552932c-8f6b-4ce4-afc3-ce61a513603d None admin] PATH_INFO: `/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:50.175847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552932c-8f6b-4ce4-afc3-ce61a513603d None admin] RBAC: Authorizing `identity:get_project(project_id=eb211c3ed5824bbea2a5667dc1fb752a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:50.177276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552932c-8f6b-4ce4-afc3-ce61a513603d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:50.179136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 541/1080] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:42:50 2022] GET /identity/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a => generated 312 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:50.185388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebad2b-370d-4731-b796-479ddae7c499 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:50.283608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebad2b-370d-4731-b796-479ddae7c499 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:50.312881 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552932c-8f6b-4ce4-afc3-ce61a513603d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:50.317722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46695de-28ed-4684-adcd-1cc7f15f7630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:50.318380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46695de-28ed-4684-adcd-1cc7f15f7630 None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:50.318657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46695de-28ed-4684-adcd-1cc7f15f7630 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:50.318930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46695de-28ed-4684-adcd-1cc7f15f7630 None admin] PATH_INFO: `/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a/users/778e0e2a2d0b4443a41b143cfd1e2d63/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:50.320874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46695de-28ed-4684-adcd-1cc7f15f7630 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb211c3ed5824bbea2a5667dc1fb752a, user_id=778e0e2a2d0b4443a41b143cfd1e2d63, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:50.322601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46695de-28ed-4684-adcd-1cc7f15f7630 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:50.340357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:50.388160 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 540/1081] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:42:50 2022] PUT /identity/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a/users/778e0e2a2d0b4443a41b143cfd1e2d63/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:42:50.419315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552932c-8f6b-4ce4-afc3-ce61a513603d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:50.457861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485d402-faf7-41ac-a186-cd938d5e4c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:50.458477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485d402-faf7-41ac-a186-cd938d5e4c97 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:50.458755 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485d402-faf7-41ac-a186-cd938d5e4c97 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:50.459050 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8485d402-faf7-41ac-a186-cd938d5e4c97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:50.535197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485d402-faf7-41ac-a186-cd938d5e4c97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:50.536644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8485d402-faf7-41ac-a186-cd938d5e4c97 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:50.563279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:50.646237 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 542/1082] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:50 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:42:50.647501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46695de-28ed-4684-adcd-1cc7f15f7630 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:50.698048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:50.751553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46695de-28ed-4684-adcd-1cc7f15f7630 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:50.788831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d952a6d-b79d-4754-ab14-7ed07057a912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:50.789459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d952a6d-b79d-4754-ab14-7ed07057a912 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:50.790535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d952a6d-b79d-4754-ab14-7ed07057a912 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:50.790807 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d952a6d-b79d-4754-ab14-7ed07057a912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:50.800872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d952a6d-b79d-4754-ab14-7ed07057a912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:50.802305 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d952a6d-b79d-4754-ab14-7ed07057a912 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-3d952a6d-b79d-4754-ab14-7ed07057a912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:50.818193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:50.819421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:50.820350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:50.820350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:50.820350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:50.820350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 541/1083] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:50.831062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485d402-faf7-41ac-a186-cd938d5e4c97 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:50.929409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8485d402-faf7-41ac-a186-cd938d5e4c97 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:50.953363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:50.964825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f2a57-9e16-43e9-a9ec-0c07a342212a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:50.965432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f2a57-9e16-43e9-a9ec-0c07a342212a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:50.967059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f2a57-9e16-43e9-a9ec-0c07a342212a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:50.967059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f2a57-9e16-43e9-a9ec-0c07a342212a None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:50.967059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f2a57-9e16-43e9-a9ec-0c07a342212a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:50.967059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f2a57-9e16-43e9-a9ec-0c07a342212a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:50.968245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f2a57-9e16-43e9-a9ec-0c07a342212a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:50.980733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 543/1084] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:51.529221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fe571-7b65-4bc0-8044-cd53bfd2f470 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:51.529751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fe571-7b65-4bc0-8044-cd53bfd2f470 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:51.529751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fe571-7b65-4bc0-8044-cd53bfd2f470 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:51.531096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 542/1085] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:51.536358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda62fc0-562d-4fe2-b564-7ac27fd1af73 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:51.536643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda62fc0-562d-4fe2-b564-7ac27fd1af73 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:51.536918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda62fc0-562d-4fe2-b564-7ac27fd1af73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:51.555401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-cda62fc0-562d-4fe2-b564-7ac27fd1af73 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:51.556640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cda62fc0-562d-4fe2-b564-7ac27fd1af73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:51.621284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 544/1086] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:51 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:51.892004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fe571-7b65-4bc0-8044-cd53bfd2f470 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:51.985140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:51.991809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fe571-7b65-4bc0-8044-cd53bfd2f470 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:52.026558 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f50c5-d090-49e8-aa35-78052f6ffda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:52.027219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f50c5-d090-49e8-aa35-78052f6ffda6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.027495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f50c5-d090-49e8-aa35-78052f6ffda6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.027762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f50c5-d090-49e8-aa35-78052f6ffda6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.037496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f50c5-d090-49e8-aa35-78052f6ffda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-7c7f50c5-d090-49e8-aa35-78052f6ffda6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:52.042395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:52.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:52.044426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 543/1087] 10.222.0.105 () {56 vars in 1125 bytes} [Wed Aug 24 04:42:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:52.049542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda62fc0-562d-4fe2-b564-7ac27fd1af73 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:52.146980 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda62fc0-562d-4fe2-b564-7ac27fd1af73 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:52.161785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:52.182941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98327e37-6cb1-45fc-b280-7ea6bfef6c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:52.183596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327e37-6cb1-45fc-b280-7ea6bfef6c34 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.183876 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327e37-6cb1-45fc-b280-7ea6bfef6c34 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.184150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98327e37-6cb1-45fc-b280-7ea6bfef6c34 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.184814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98327e37-6cb1-45fc-b280-7ea6bfef6c34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:52.185116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98327e37-6cb1-45fc-b280-7ea6bfef6c34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:52.186584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98327e37-6cb1-45fc-b280-7ea6bfef6c34 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:52.195345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 545/1088] 10.222.0.105 () {56 vars in 1120 bytes} [Wed Aug 24 04:42:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:52.201985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f50c5-d090-49e8-aa35-78052f6ffda6 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:52.299608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f50c5-d090-49e8-aa35-78052f6ffda6 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:52.310207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d13329-7045-40c9-9454-0aec43784a25 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.310593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d13329-7045-40c9-9454-0aec43784a25 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.310878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d13329-7045-40c9-9454-0aec43784a25 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.312252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 546/1089] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:42:52.317632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1808af-33ed-4716-ae5f-452264bbbefa None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.317896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1808af-33ed-4716-ae5f-452264bbbefa None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.318171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1808af-33ed-4716-ae5f-452264bbbefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.333344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178b91-1df9-4a98-9524-f7a9079b5304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:52.333964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178b91-1df9-4a98-9524-f7a9079b5304 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.334241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178b91-1df9-4a98-9524-f7a9079b5304 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.334511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0178b91-1df9-4a98-9524-f7a9079b5304 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.335714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-4c1808af-33ed-4716-ae5f-452264bbbefa None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:52.336939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4c1808af-33ed-4716-ae5f-452264bbbefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:52.345061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0178b91-1df9-4a98-9524-f7a9079b5304 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:52.346489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0178b91-1df9-4a98-9524-f7a9079b5304 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-e0178b91-1df9-4a98-9524-f7a9079b5304 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:52.360820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:52.361805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:52.363199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:52.363199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:52.363199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:52.363199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 544/1090] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:52.367651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178b91-1df9-4a98-9524-f7a9079b5304 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:52.378816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:52.406956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 547/1091] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:52 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:52.466217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0178b91-1df9-4a98-9524-f7a9079b5304 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:52.499485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9144882-9520-4998-8f37-910a80dde45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:52.500080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9144882-9520-4998-8f37-910a80dde45e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.500355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9144882-9520-4998-8f37-910a80dde45e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.500622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9144882-9520-4998-8f37-910a80dde45e None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.501243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9144882-9520-4998-8f37-910a80dde45e None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:52.501536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9144882-9520-4998-8f37-910a80dde45e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:52.502878 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9144882-9520-4998-8f37-910a80dde45e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:52.514754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 545/1092] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:52.520331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1808af-33ed-4716-ae5f-452264bbbefa None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:52.617637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1808af-33ed-4716-ae5f-452264bbbefa None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:52.624901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:52.651803 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306619e7-c380-4687-b489-3f0b69e74e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:52.652423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306619e7-c380-4687-b489-3f0b69e74e0d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.655400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306619e7-c380-4687-b489-3f0b69e74e0d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.655400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306619e7-c380-4687-b489-3f0b69e74e0d None admin] PATH_INFO: `/v3/users/nova` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.660096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306619e7-c380-4687-b489-3f0b69e74e0d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:52.661497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306619e7-c380-4687-b489-3f0b69e74e0d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-306619e7-c380-4687-b489-3f0b69e74e0d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:42:52.673889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:52.675640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:52.676727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 548/1093] 10.222.0.105 () {56 vars in 1098 bytes} [Wed Aug 24 04:42:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:52.677326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9144882-9520-4998-8f37-910a80dde45e None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:52.681178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306619e7-c380-4687-b489-3f0b69e74e0d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:52.745332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:52.778931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9144882-9520-4998-8f37-910a80dde45e None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:52.780031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306619e7-c380-4687-b489-3f0b69e74e0d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:52.814932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407c2c5-04a5-4e9a-8765-eccd2b10ac41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:52.815370 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e64672-b1ed-4d27-a0eb-0137121f6f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:52.815584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407c2c5-04a5-4e9a-8765-eccd2b10ac41 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.815858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407c2c5-04a5-4e9a-8765-eccd2b10ac41 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.815961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e64672-b1ed-4d27-a0eb-0137121f6f0b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.816125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407c2c5-04a5-4e9a-8765-eccd2b10ac41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.816226 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e64672-b1ed-4d27-a0eb-0137121f6f0b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.816492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e64672-b1ed-4d27-a0eb-0137121f6f0b None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.817207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e64672-b1ed-4d27-a0eb-0137121f6f0b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:52.817510 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e64672-b1ed-4d27-a0eb-0137121f6f0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:52.818924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e64672-b1ed-4d27-a0eb-0137121f6f0b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:52.826454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4407c2c5-04a5-4e9a-8765-eccd2b10ac41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-4407c2c5-04a5-4e9a-8765-eccd2b10ac41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:42:52.831141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:52.832178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:42:52.833041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 546/1094] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:42:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:52.839102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 549/1095] 10.222.0.105 () {56 vars in 1138 bytes} [Wed Aug 24 04:42:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:52.839249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407c2c5-04a5-4e9a-8765-eccd2b10ac41 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:52.843320 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e64672-b1ed-4d27-a0eb-0137121f6f0b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:52.867534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:52.938009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407c2c5-04a5-4e9a-8765-eccd2b10ac41 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:52.941934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e64672-b1ed-4d27-a0eb-0137121f6f0b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:52.963915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:52.973918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bff1e-d8ae-4353-b85f-dffb9d91fc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:52.974787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bff1e-d8ae-4353-b85f-dffb9d91fc6f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.975284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bff1e-d8ae-4353-b85f-dffb9d91fc6f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.975729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bff1e-d8ae-4353-b85f-dffb9d91fc6f None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.976828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bff1e-d8ae-4353-b85f-dffb9d91fc6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:52.977918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bff1e-d8ae-4353-b85f-dffb9d91fc6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:52.980245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bff1e-d8ae-4353-b85f-dffb9d91fc6f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:52.982799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe5c5a-cd43-444e-828d-4e4d6e0cc09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:52.983664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe5c5a-cd43-444e-828d-4e4d6e0cc09c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:52.984110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe5c5a-cd43-444e-828d-4e4d6e0cc09c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:52.984555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe5c5a-cd43-444e-828d-4e4d6e0cc09c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:52.998310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 547/1096] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:42:52 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:53.000459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bff1e-d8ae-4353-b85f-dffb9d91fc6f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:53.057328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe5c5a-cd43-444e-828d-4e4d6e0cc09c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:53.058796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe5c5a-cd43-444e-828d-4e4d6e0cc09c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:53.082786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:53.097791 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bff1e-d8ae-4353-b85f-dffb9d91fc6f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:53.130684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7185-be73-45ff-b9bd-750402865492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:53.131338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7185-be73-45ff-b9bd-750402865492 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:53.131622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7185-be73-45ff-b9bd-750402865492 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:53.131889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c7185-be73-45ff-b9bd-750402865492 None admin] PATH_INFO: `/v3/users/778e0e2a2d0b4443a41b143cfd1e2d63` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:53.133008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c7185-be73-45ff-b9bd-750402865492 None admin] RBAC: Authorizing `identity:get_user(user_id=778e0e2a2d0b4443a41b143cfd1e2d63)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:53.134931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c7185-be73-45ff-b9bd-750402865492 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:53.136516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 548/1097] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:42:52 2022] GET /identity/v3/users/778e0e2a2d0b4443a41b143cfd1e2d63 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:53.142415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7185-be73-45ff-b9bd-750402865492 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:53.162114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 550/1098] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:52 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:42:53.167960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe5c5a-cd43-444e-828d-4e4d6e0cc09c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:53.199835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:53.240362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c7185-be73-45ff-b9bd-750402865492 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:53.273667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cef53-0e51-4845-956a-45a614744e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:53.274827 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cef53-0e51-4845-956a-45a614744e7b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:53.274944 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe5c5a-cd43-444e-828d-4e4d6e0cc09c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:53.275128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cef53-0e51-4845-956a-45a614744e7b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:53.275400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cef53-0e51-4845-956a-45a614744e7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:53.298392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:53.309060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245beb8-d7f6-4aea-9674-c3351a3e9896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:53.309850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245beb8-d7f6-4aea-9674-c3351a3e9896 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:53.310126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245beb8-d7f6-4aea-9674-c3351a3e9896 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:53.310394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245beb8-d7f6-4aea-9674-c3351a3e9896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:53.321811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245beb8-d7f6-4aea-9674-c3351a3e9896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:53.323462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245beb8-d7f6-4aea-9674-c3351a3e9896 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-3245beb8-d7f6-4aea-9674-c3351a3e9896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:42:53.338176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:53.339271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:42:53.340752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:42:53.340752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:42:53.340752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:53.340752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 551/1099] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:42:53.344583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cef53-0e51-4845-956a-45a614744e7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:53.345025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245beb8-d7f6-4aea-9674-c3351a3e9896 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:53.346173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cef53-0e51-4845-956a-45a614744e7b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:53.401934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:53.443864 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245beb8-d7f6-4aea-9674-c3351a3e9896 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:53.444797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 549/1100] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:53 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 04:42:53.451233 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cef53-0e51-4845-956a-45a614744e7b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:53.476994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636a6e3-b4c2-4e58-9bea-518059f93893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:53.477616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636a6e3-b4c2-4e58-9bea-518059f93893 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:53.477894 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636a6e3-b4c2-4e58-9bea-518059f93893 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:53.478164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636a6e3-b4c2-4e58-9bea-518059f93893 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:53.478793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636a6e3-b4c2-4e58-9bea-518059f93893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:53.479116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636a6e3-b4c2-4e58-9bea-518059f93893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:53.480462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636a6e3-b4c2-4e58-9bea-518059f93893 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:53.491911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 552/1101] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:42:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:53.498232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636a6e3-b4c2-4e58-9bea-518059f93893 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:53.532126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:53.547585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cef53-0e51-4845-956a-45a614744e7b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:53.582632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695d44c-539b-40f6-8b3b-8df4d58ce648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:53.583280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695d44c-539b-40f6-8b3b-8df4d58ce648 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:53.583553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695d44c-539b-40f6-8b3b-8df4d58ce648 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:53.583817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695d44c-539b-40f6-8b3b-8df4d58ce648 None admin] PATH_INFO: `/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:53.584912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695d44c-539b-40f6-8b3b-8df4d58ce648 None admin] RBAC: Authorizing `identity:get_project(project_id=eb211c3ed5824bbea2a5667dc1fb752a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:53.586315 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695d44c-539b-40f6-8b3b-8df4d58ce648 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:53.588521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 550/1102] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:42:53 2022] GET /identity/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:53.594467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695d44c-539b-40f6-8b3b-8df4d58ce648 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:53.595099 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636a6e3-b4c2-4e58-9bea-518059f93893 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:53.621674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:53.631867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590333b-afcd-4fab-bf33-e90f1e1213c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:53.643583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590333b-afcd-4fab-bf33-e90f1e1213c1 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:53.643583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590333b-afcd-4fab-bf33-e90f1e1213c1 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:53.643583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590333b-afcd-4fab-bf33-e90f1e1213c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:53.643583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590333b-afcd-4fab-bf33-e90f1e1213c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:53.643583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590333b-afcd-4fab-bf33-e90f1e1213c1 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-1590333b-afcd-4fab-bf33-e90f1e1213c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:42:53.652932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:42:53.653862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:42:53.654812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:42:53.654812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:42:53.654812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:42:53.654812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:42:53.654812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:42:53.654812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:42:53.654812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:42:53.654812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:42:53.654812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 553/1103] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:42:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:42:53.659994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590333b-afcd-4fab-bf33-e90f1e1213c1 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:53.693647 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695d44c-539b-40f6-8b3b-8df4d58ce648 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:53.726821 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d3613-cdd5-424a-aa1b-400d138e49c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:53.727794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d3613-cdd5-424a-aa1b-400d138e49c7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:53.728078 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d3613-cdd5-424a-aa1b-400d138e49c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:53.728341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d3613-cdd5-424a-aa1b-400d138e49c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:53.729061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d3613-cdd5-424a-aa1b-400d138e49c7 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=eb211c3ed5824bbea2a5667dc1fb752a, user.id=778e0e2a2d0b4443a41b143cfd1e2d63) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:53.729361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d3613-cdd5-424a-aa1b-400d138e49c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:53.730821 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d3613-cdd5-424a-aa1b-400d138e49c7 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:53.737392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 551/1104] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:53 2022] GET /identity/v3/role_assignments?user.id=778e0e2a2d0b4443a41b143cfd1e2d63&scope.project.id=eb211c3ed5824bbea2a5667dc1fb752a&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 610 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:53.754083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:53.761178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590333b-afcd-4fab-bf33-e90f1e1213c1 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:53.799301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e188ca-7f38-4aa1-801b-ee904d722d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:53.799922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e188ca-7f38-4aa1-801b-ee904d722d88 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:53.800198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e188ca-7f38-4aa1-801b-ee904d722d88 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:53.800469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e188ca-7f38-4aa1-801b-ee904d722d88 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:53.801175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e188ca-7f38-4aa1-801b-ee904d722d88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:53.801477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e188ca-7f38-4aa1-801b-ee904d722d88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:53.802926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e188ca-7f38-4aa1-801b-ee904d722d88 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:53.822784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 554/1105] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:42:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:42:53.824522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d3613-cdd5-424a-aa1b-400d138e49c7 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:53.918688 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:53.927854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d3613-cdd5-424a-aa1b-400d138e49c7 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:53.962492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e517e33-6ae0-4e06-adbf-ef49c449e6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:53.963127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517e33-6ae0-4e06-adbf-ef49c449e6fe None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:53.963409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517e33-6ae0-4e06-adbf-ef49c449e6fe None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:53.963683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e517e33-6ae0-4e06-adbf-ef49c449e6fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:53.964397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e517e33-6ae0-4e06-adbf-ef49c449e6fe None admin] RBAC: Adding query filter params (role.id=dced619d94db43ed8cccb9c80be594b9, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=3c822c8f23b74bae84769046fdb85db4) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:53.964726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e517e33-6ae0-4e06-adbf-ef49c449e6fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:53.966161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e517e33-6ae0-4e06-adbf-ef49c449e6fe None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:53.971834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 552/1106] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:53 2022] GET /identity/v3/role_assignments?user.id=3c822c8f23b74bae84769046fdb85db4&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=dced619d94db43ed8cccb9c80be594b9 => generated 610 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:55.681505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17343f2e-bd26-4991-a9c9-02c5d1910b70 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:55.682032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17343f2e-bd26-4991-a9c9-02c5d1910b70 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:55.682032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17343f2e-bd26-4991-a9c9-02c5d1910b70 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:55.683412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 555/1107] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:55 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:55.688505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639d951-7dc5-4875-b75e-ffc4dd836504 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:55.689171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639d951-7dc5-4875-b75e-ffc4dd836504 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:55.689304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639d951-7dc5-4875-b75e-ffc4dd836504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:55.707291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5639d951-7dc5-4875-b75e-ffc4dd836504 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:55.708449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5639d951-7dc5-4875-b75e-ffc4dd836504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:55.771508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 553/1108] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:55 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:42:56.043133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17343f2e-bd26-4991-a9c9-02c5d1910b70 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:56.224760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17343f2e-bd26-4991-a9c9-02c5d1910b70 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:56.258217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662d41d-5534-4a34-bfd2-70e7a971defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:56.258873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d41d-5534-4a34-bfd2-70e7a971defa None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:56.259175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d41d-5534-4a34-bfd2-70e7a971defa None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:56.259447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662d41d-5534-4a34-bfd2-70e7a971defa None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:56.260546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662d41d-5534-4a34-bfd2-70e7a971defa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:56.261967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662d41d-5534-4a34-bfd2-70e7a971defa None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:56.263539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 556/1109] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:42:56 2022] GET /identity/v3/domains/default => generated 200 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:56.269340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639d951-7dc5-4875-b75e-ffc4dd836504 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:56.351894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:56.367569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639d951-7dc5-4875-b75e-ffc4dd836504 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:56.401970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576ade99-0a10-431a-adac-8d3b4d349830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:56.402568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ade99-0a10-431a-adac-8d3b4d349830 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:56.402845 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ade99-0a10-431a-adac-8d3b4d349830 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:56.403142 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576ade99-0a10-431a-adac-8d3b4d349830 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:56.403789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ade99-0a10-431a-adac-8d3b4d349830 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:56.405337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576ade99-0a10-431a-adac-8d3b4d349830 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:56.485666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 554/1110] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:42:56 2022] POST /identity/v3/users => generated 287 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:42:58.429936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc913d-4f55-44f7-bb39-5905e757ec37 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:58.431406 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc913d-4f55-44f7-bb39-5905e757ec37 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:58.431582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc913d-4f55-44f7-bb39-5905e757ec37 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:58.432801 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 557/1111] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:42:58 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:42:58.438379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965ebcb-f638-484b-807c-4ece2dc737d5 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:58.438778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965ebcb-f638-484b-807c-4ece2dc737d5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:58.439221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965ebcb-f638-484b-807c-4ece2dc737d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:58.456934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-0965ebcb-f638-484b-807c-4ece2dc737d5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:42:58.458040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0965ebcb-f638-484b-807c-4ece2dc737d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:42:58.522301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 555/1112] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:42:58 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:42:58.794363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc913d-4f55-44f7-bb39-5905e757ec37 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:58.819047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:58.893680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc913d-4f55-44f7-bb39-5905e757ec37 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:58.926883 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf0a2f-7bde-44c4-84d4-09b1f6995ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:58.927525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf0a2f-7bde-44c4-84d4-09b1f6995ac2 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:58.927804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf0a2f-7bde-44c4-84d4-09b1f6995ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:58.928076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf0a2f-7bde-44c4-84d4-09b1f6995ac2 None admin] PATH_INFO: `/v3/roles/b7f17471d2c948efa07d02424da68bea` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:58.929457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf0a2f-7bde-44c4-84d4-09b1f6995ac2 None admin] RBAC: Authorizing `identity:get_role(role_id=b7f17471d2c948efa07d02424da68bea)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:58.930860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf0a2f-7bde-44c4-84d4-09b1f6995ac2 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:58.932179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 558/1113] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:42:58 2022] GET /identity/v3/roles/b7f17471d2c948efa07d02424da68bea => generated 224 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:58.937883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965ebcb-f638-484b-807c-4ece2dc737d5 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:59.007112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:59.036130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965ebcb-f638-484b-807c-4ece2dc737d5 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:59.072109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfdef0-07cb-4a5e-a82a-de938b715199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:59.072672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfdef0-07cb-4a5e-a82a-de938b715199 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:59.072949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfdef0-07cb-4a5e-a82a-de938b715199 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:59.073217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfdef0-07cb-4a5e-a82a-de938b715199 None admin] PATH_INFO: `/v3/users/48d57f774e8a4408b41a573722539675` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:59.087593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfdef0-07cb-4a5e-a82a-de938b715199 None admin] RBAC: Authorizing `identity:get_user(user_id=48d57f774e8a4408b41a573722539675)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:59.089155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfdef0-07cb-4a5e-a82a-de938b715199 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:59.090748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 556/1114] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:42:58 2022] GET /identity/v3/users/48d57f774e8a4408b41a573722539675 => generated 287 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:59.096827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf0a2f-7bde-44c4-84d4-09b1f6995ac2 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:59.186837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:59.196414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf0a2f-7bde-44c4-84d4-09b1f6995ac2 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:59.229826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f51b3b-4131-4706-be76-4558cbdcd75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:59.230417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f51b3b-4131-4706-be76-4558cbdcd75d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:59.230689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f51b3b-4131-4706-be76-4558cbdcd75d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:59.230955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f51b3b-4131-4706-be76-4558cbdcd75d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:59.296858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f51b3b-4131-4706-be76-4558cbdcd75d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:59.298279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f51b3b-4131-4706-be76-4558cbdcd75d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:59.387175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:59.396653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 559/1115] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:42:59 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:42:59.403849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfdef0-07cb-4a5e-a82a-de938b715199 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:59.502378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfdef0-07cb-4a5e-a82a-de938b715199 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:59.529216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:42:59.536723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11582465-ecf7-469b-9e9b-4f5f7050e4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:59.537342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11582465-ecf7-469b-9e9b-4f5f7050e4c9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:59.537616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11582465-ecf7-469b-9e9b-4f5f7050e4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:59.537884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11582465-ecf7-469b-9e9b-4f5f7050e4c9 None admin] PATH_INFO: `/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:59.538997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11582465-ecf7-469b-9e9b-4f5f7050e4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=eb211c3ed5824bbea2a5667dc1fb752a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:59.540768 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11582465-ecf7-469b-9e9b-4f5f7050e4c9 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:59.542420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 557/1116] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:42:59 2022] GET /identity/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a => generated 312 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:42:59.548580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f51b3b-4131-4706-be76-4558cbdcd75d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:42:59.644987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f51b3b-4131-4706-be76-4558cbdcd75d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:42:59.676966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9856a6-0c87-4dee-abf6-e2419b298726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:42:59.677578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9856a6-0c87-4dee-abf6-e2419b298726 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:42:59.677855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9856a6-0c87-4dee-abf6-e2419b298726 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:42:59.678119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9856a6-0c87-4dee-abf6-e2419b298726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:42:59.678827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9856a6-0c87-4dee-abf6-e2419b298726 None admin] RBAC: Adding query filter params (role.id=b7f17471d2c948efa07d02424da68bea, scope.project.id=eb211c3ed5824bbea2a5667dc1fb752a, user.id=48d57f774e8a4408b41a573722539675) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:42:59.679155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9856a6-0c87-4dee-abf6-e2419b298726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:42:59.681365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9856a6-0c87-4dee-abf6-e2419b298726 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:42:59.686475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 560/1117] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:42:59 2022] GET /identity/v3/role_assignments?user.id=48d57f774e8a4408b41a573722539675&scope.project.id=eb211c3ed5824bbea2a5667dc1fb752a&role.id=b7f17471d2c948efa07d02424da68bea => generated 260 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:01.539046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64712683-23c9-454c-a1ef-becd546e30d6 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:01.539556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64712683-23c9-454c-a1ef-becd546e30d6 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:01.539556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64712683-23c9-454c-a1ef-becd546e30d6 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:01.541190 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 558/1118] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:01.546279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6107b-e658-43c6-9a59-6c543b6b7f41 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:01.546715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6107b-e658-43c6-9a59-6c543b6b7f41 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:01.546818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6107b-e658-43c6-9a59-6c543b6b7f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:01.565455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-aae6107b-e658-43c6-9a59-6c543b6b7f41 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:01.566635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-aae6107b-e658-43c6-9a59-6c543b6b7f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:01.615197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:01.630768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 561/1119] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:01 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:01.907455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64712683-23c9-454c-a1ef-becd546e30d6 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:02.005253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64712683-23c9-454c-a1ef-becd546e30d6 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:02.037931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee46aea-e412-451b-a471-1c1ff5236ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:02.038544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee46aea-e412-451b-a471-1c1ff5236ed4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:02.038823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee46aea-e412-451b-a471-1c1ff5236ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:02.039123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee46aea-e412-451b-a471-1c1ff5236ed4 None admin] PATH_INFO: `/v3/roles/b7f17471d2c948efa07d02424da68bea` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:02.041157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee46aea-e412-451b-a471-1c1ff5236ed4 None admin] RBAC: Authorizing `identity:get_role(role_id=b7f17471d2c948efa07d02424da68bea)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:02.042506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee46aea-e412-451b-a471-1c1ff5236ed4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:02.043830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 559/1120] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:43:01 2022] GET /identity/v3/roles/b7f17471d2c948efa07d02424da68bea => generated 224 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:02.049579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6107b-e658-43c6-9a59-6c543b6b7f41 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:02.144942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6107b-e658-43c6-9a59-6c543b6b7f41 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:02.177805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2fc34-a4b8-4b6d-adde-8a9af12346d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:02.178367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2fc34-a4b8-4b6d-adde-8a9af12346d3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:02.178643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2fc34-a4b8-4b6d-adde-8a9af12346d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:02.178909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2fc34-a4b8-4b6d-adde-8a9af12346d3 None admin] PATH_INFO: `/v3/users/48d57f774e8a4408b41a573722539675` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:02.180038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2fc34-a4b8-4b6d-adde-8a9af12346d3 None admin] RBAC: Authorizing `identity:get_user(user_id=48d57f774e8a4408b41a573722539675)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:02.182255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c2fc34-a4b8-4b6d-adde-8a9af12346d3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:02.183848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 562/1121] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:43:02 2022] GET /identity/v3/users/48d57f774e8a4408b41a573722539675 => generated 287 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:02.189832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee46aea-e412-451b-a471-1c1ff5236ed4 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:02.250177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:02.285234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee46aea-e412-451b-a471-1c1ff5236ed4 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:02.319182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fa7f67-0582-461c-b4c3-e508ef5272aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:02.319795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa7f67-0582-461c-b4c3-e508ef5272aa None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:02.320072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa7f67-0582-461c-b4c3-e508ef5272aa None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:02.320340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa7f67-0582-461c-b4c3-e508ef5272aa None admin] PATH_INFO: `/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:02.321530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa7f67-0582-461c-b4c3-e508ef5272aa None admin] RBAC: Authorizing `identity:get_project(project_id=eb211c3ed5824bbea2a5667dc1fb752a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:02.322931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fa7f67-0582-461c-b4c3-e508ef5272aa None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:02.324558 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 560/1122] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:43:02 2022] GET /identity/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a => generated 312 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:02.330422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2fc34-a4b8-4b6d-adde-8a9af12346d3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:02.385437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:02.428997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2fc34-a4b8-4b6d-adde-8a9af12346d3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:02.460797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84324ab9-13ad-474f-9cf3-9d112df0929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:02.461923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324ab9-13ad-474f-9cf3-9d112df0929a None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:02.462310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324ab9-13ad-474f-9cf3-9d112df0929a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:02.462725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84324ab9-13ad-474f-9cf3-9d112df0929a None admin] PATH_INFO: `/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a/users/48d57f774e8a4408b41a573722539675/roles/b7f17471d2c948efa07d02424da68bea` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:02.464926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84324ab9-13ad-474f-9cf3-9d112df0929a None admin] RBAC: Authorizing `identity:create_grant(project_id=eb211c3ed5824bbea2a5667dc1fb752a, user_id=48d57f774e8a4408b41a573722539675, role_id=b7f17471d2c948efa07d02424da68bea)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:02.466323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84324ab9-13ad-474f-9cf3-9d112df0929a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:02.542335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 563/1123] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:43:02 2022] PUT /identity/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a/users/48d57f774e8a4408b41a573722539675/roles/b7f17471d2c948efa07d02424da68bea => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:43:04.424807 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37a61-49ee-4c71-87f1-891dca2672c2 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:04.425343 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37a61-49ee-4c71-87f1-891dca2672c2 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:04.425343 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f37a61-49ee-4c71-87f1-891dca2672c2 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:04.426674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 561/1124] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:04.432730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d18a-f4b9-4363-bfff-928a7c7a9aad None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:04.433124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d18a-f4b9-4363-bfff-928a7c7a9aad None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:04.433267 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692d18a-f4b9-4363-bfff-928a7c7a9aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:04.451427 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b692d18a-f4b9-4363-bfff-928a7c7a9aad None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:04.452545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b692d18a-f4b9-4363-bfff-928a7c7a9aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:04.517155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 564/1125] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:04 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:43:04.792031 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f37a61-49ee-4c71-87f1-891dca2672c2 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:04.889411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f37a61-49ee-4c71-87f1-891dca2672c2 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:04.919406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:04.924117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dde8ff-7d8a-4ed5-8a99-29826b0d11f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:04.924772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dde8ff-7d8a-4ed5-8a99-29826b0d11f1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:04.925051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dde8ff-7d8a-4ed5-8a99-29826b0d11f1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:04.925322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dde8ff-7d8a-4ed5-8a99-29826b0d11f1 None admin] PATH_INFO: `/v3/roles/b7f17471d2c948efa07d02424da68bea` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:04.926702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dde8ff-7d8a-4ed5-8a99-29826b0d11f1 None admin] RBAC: Authorizing `identity:get_role(role_id=b7f17471d2c948efa07d02424da68bea)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:04.928094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dde8ff-7d8a-4ed5-8a99-29826b0d11f1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:04.929397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 562/1126] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:43:04 2022] GET /identity/v3/roles/b7f17471d2c948efa07d02424da68bea => generated 224 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:04.935255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d18a-f4b9-4363-bfff-928a7c7a9aad None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:04.981343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:05.034302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692d18a-f4b9-4363-bfff-928a7c7a9aad None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:05.067921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe068998-4d13-4733-9574-f3cd20207a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:05.068530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe068998-4d13-4733-9574-f3cd20207a83 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:05.068804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe068998-4d13-4733-9574-f3cd20207a83 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:05.069071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe068998-4d13-4733-9574-f3cd20207a83 None admin] PATH_INFO: `/v3/users/48d57f774e8a4408b41a573722539675` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:05.070164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe068998-4d13-4733-9574-f3cd20207a83 None admin] RBAC: Authorizing `identity:get_user(user_id=48d57f774e8a4408b41a573722539675)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:05.072360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe068998-4d13-4733-9574-f3cd20207a83 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:05.073902 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 565/1127] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:43:04 2022] GET /identity/v3/users/48d57f774e8a4408b41a573722539675 => generated 287 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:05.079653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dde8ff-7d8a-4ed5-8a99-29826b0d11f1 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:05.176656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dde8ff-7d8a-4ed5-8a99-29826b0d11f1 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:05.211180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f66f4-6967-42d6-b30d-a2f4a4b60baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:05.212498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f66f4-6967-42d6-b30d-a2f4a4b60baf None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:05.212748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f66f4-6967-42d6-b30d-a2f4a4b60baf None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:05.213019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f66f4-6967-42d6-b30d-a2f4a4b60baf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:05.275854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:05.278077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f66f4-6967-42d6-b30d-a2f4a4b60baf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:05.279552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f66f4-6967-42d6-b30d-a2f4a4b60baf None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:05.373837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 563/1128] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:43:05 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:43:05.379866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe068998-4d13-4733-9574-f3cd20207a83 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:05.482796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe068998-4d13-4733-9574-f3cd20207a83 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:05.495568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:05.516362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132a0b3-6c74-4614-9ab7-d02b8f8c971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:05.516978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132a0b3-6c74-4614-9ab7-d02b8f8c971e None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:05.517251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132a0b3-6c74-4614-9ab7-d02b8f8c971e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:05.517516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132a0b3-6c74-4614-9ab7-d02b8f8c971e None admin] PATH_INFO: `/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:05.518648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132a0b3-6c74-4614-9ab7-d02b8f8c971e None admin] RBAC: Authorizing `identity:get_project(project_id=eb211c3ed5824bbea2a5667dc1fb752a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:05.520084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132a0b3-6c74-4614-9ab7-d02b8f8c971e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:05.521673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 566/1129] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:43:05 2022] GET /identity/v3/projects/eb211c3ed5824bbea2a5667dc1fb752a => generated 312 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:05.527942 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f66f4-6967-42d6-b30d-a2f4a4b60baf None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:05.620285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:05.623096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f66f4-6967-42d6-b30d-a2f4a4b60baf None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:05.655650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff49d89-ef3c-453c-b506-550176c7d2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:05.656181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff49d89-ef3c-453c-b506-550176c7d2fa None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:05.656457 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff49d89-ef3c-453c-b506-550176c7d2fa None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:05.656726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff49d89-ef3c-453c-b506-550176c7d2fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:05.657438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff49d89-ef3c-453c-b506-550176c7d2fa None admin] RBAC: Adding query filter params (role.id=b7f17471d2c948efa07d02424da68bea, scope.project.id=eb211c3ed5824bbea2a5667dc1fb752a, user.id=48d57f774e8a4408b41a573722539675) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:05.657740 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff49d89-ef3c-453c-b506-550176c7d2fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:05.659190 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff49d89-ef3c-453c-b506-550176c7d2fa None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:05.664714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 564/1130] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:43:05 2022] GET /identity/v3/role_assignments?user.id=48d57f774e8a4408b41a573722539675&scope.project.id=eb211c3ed5824bbea2a5667dc1fb752a&role.id=b7f17471d2c948efa07d02424da68bea => generated 610 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:07.576791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb1f14-8b50-4f46-b998-1cb76d59785e None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:07.577326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb1f14-8b50-4f46-b998-1cb76d59785e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:07.577326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb1f14-8b50-4f46-b998-1cb76d59785e None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:07.578685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 567/1131] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:07.584042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e3606-965a-40fb-8512-6d080703e950 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:07.584448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e3606-965a-40fb-8512-6d080703e950 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:07.584579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e3606-965a-40fb-8512-6d080703e950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:07.603407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-0a5e3606-965a-40fb-8512-6d080703e950 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:07.604583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0a5e3606-965a-40fb-8512-6d080703e950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:07.669586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 565/1132] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:07 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:43:07.936184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb1f14-8b50-4f46-b998-1cb76d59785e None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:08.036222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb1f14-8b50-4f46-b998-1cb76d59785e None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:08.070071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c515a9f-56a2-42e5-a219-9eca0d4b86ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:08.070678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c515a9f-56a2-42e5-a219-9eca0d4b86ef None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:08.071105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c515a9f-56a2-42e5-a219-9eca0d4b86ef None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:08.071567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c515a9f-56a2-42e5-a219-9eca0d4b86ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:08.072923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c515a9f-56a2-42e5-a219-9eca0d4b86ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:08.074363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c515a9f-56a2-42e5-a219-9eca0d4b86ef None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:08.075937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 568/1133] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:43:07 2022] GET /identity/v3/domains/default => generated 200 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:08.081797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e3606-965a-40fb-8512-6d080703e950 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:08.171846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:08.178096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e3606-965a-40fb-8512-6d080703e950 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:08.211798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555d660b-fbcd-45c0-a0a4-ecf6396f7cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:08.212400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d660b-fbcd-45c0-a0a4-ecf6396f7cb8 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:08.212674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d660b-fbcd-45c0-a0a4-ecf6396f7cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:08.212937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d660b-fbcd-45c0-a0a4-ecf6396f7cb8 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:08.213575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d660b-fbcd-45c0-a0a4-ecf6396f7cb8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:08.214975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555d660b-fbcd-45c0-a0a4-ecf6396f7cb8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:08.321443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 566/1134] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:43:08 2022] POST /identity/v3/projects => generated 312 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:43:10.242243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df63239-194a-426f-8bb5-57952eeee219 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:10.242735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df63239-194a-426f-8bb5-57952eeee219 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:10.242842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df63239-194a-426f-8bb5-57952eeee219 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:10.245303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 569/1135] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:10.250533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed4b8c-c23f-4f02-b0bf-a3e8f2fac416 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:10.250950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed4b8c-c23f-4f02-b0bf-a3e8f2fac416 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:10.251113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed4b8c-c23f-4f02-b0bf-a3e8f2fac416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:10.268713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-27ed4b8c-c23f-4f02-b0bf-a3e8f2fac416 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:10.269891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-27ed4b8c-c23f-4f02-b0bf-a3e8f2fac416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:10.338773 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 567/1136] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:10 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:10.609319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df63239-194a-426f-8bb5-57952eeee219 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:10.658804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:10.709301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df63239-194a-426f-8bb5-57952eeee219 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:10.741693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fae3a-0dcb-43e0-b808-6949f2285b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:10.742306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fae3a-0dcb-43e0-b808-6949f2285b12 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:10.742585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fae3a-0dcb-43e0-b808-6949f2285b12 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:10.743403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fae3a-0dcb-43e0-b808-6949f2285b12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:10.744521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fae3a-0dcb-43e0-b808-6949f2285b12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:10.745937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11fae3a-0dcb-43e0-b808-6949f2285b12 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:10.747533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 570/1137] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:43:10 2022] GET /identity/v3/domains/default => generated 200 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:10.753212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed4b8c-c23f-4f02-b0bf-a3e8f2fac416 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:10.827527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:10.848779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed4b8c-c23f-4f02-b0bf-a3e8f2fac416 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:10.882957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a5266-9c71-467e-a08e-91e216c957ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:10.884113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a5266-9c71-467e-a08e-91e216c957ee None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:10.884637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a5266-9c71-467e-a08e-91e216c957ee None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:10.884945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a5266-9c71-467e-a08e-91e216c957ee None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:10.885574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a5266-9c71-467e-a08e-91e216c957ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:10.886969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a5266-9c71-467e-a08e-91e216c957ee None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:10.960051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 568/1138] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:43:10 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:43:12.903395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609110c-faeb-4c2b-af76-9299006c4170 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:12.903963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609110c-faeb-4c2b-af76-9299006c4170 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:12.903963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609110c-faeb-4c2b-af76-9299006c4170 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:12.905292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 571/1139] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:12.910445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb30876-4884-45e8-b967-cd3c93a3c516 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:12.910810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb30876-4884-45e8-b967-cd3c93a3c516 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:12.910998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb30876-4884-45e8-b967-cd3c93a3c516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:12.928653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-fbb30876-4884-45e8-b967-cd3c93a3c516 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:12.929850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-fbb30876-4884-45e8-b967-cd3c93a3c516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:12.995996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 569/1140] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:12 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:13.275714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609110c-faeb-4c2b-af76-9299006c4170 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:13.375719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609110c-faeb-4c2b-af76-9299006c4170 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:13.391732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:13.408698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77a3e0-072c-4983-9e55-3e30592e2da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:13.409318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77a3e0-072c-4983-9e55-3e30592e2da7 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:13.409597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77a3e0-072c-4983-9e55-3e30592e2da7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:13.409866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77a3e0-072c-4983-9e55-3e30592e2da7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:13.420397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77a3e0-072c-4983-9e55-3e30592e2da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-7c77a3e0-072c-4983-9e55-3e30592e2da7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:43:13.425453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:43:13.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:43:13.427717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 572/1141] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:43:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:43:13.433551 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb30876-4884-45e8-b967-cd3c93a3c516 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:13.491425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:13.529000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb30876-4884-45e8-b967-cd3c93a3c516 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:13.562300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c3076-00f7-49f5-b6a2-871c2e6c5924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:13.562894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c3076-00f7-49f5-b6a2-871c2e6c5924 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:13.563207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c3076-00f7-49f5-b6a2-871c2e6c5924 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:13.563478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c3076-00f7-49f5-b6a2-871c2e6c5924 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:13.564128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c3076-00f7-49f5-b6a2-871c2e6c5924 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:13.564677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c3076-00f7-49f5-b6a2-871c2e6c5924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:13.566017 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c3076-00f7-49f5-b6a2-871c2e6c5924 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:13.573857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 570/1142] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:43:13 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:13.579742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77a3e0-072c-4983-9e55-3e30592e2da7 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:13.677370 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77a3e0-072c-4983-9e55-3e30592e2da7 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:13.709885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f228e-49ea-4080-b991-50d578aa03b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:13.710497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f228e-49ea-4080-b991-50d578aa03b3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:13.710776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f228e-49ea-4080-b991-50d578aa03b3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:13.711068 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f228e-49ea-4080-b991-50d578aa03b3 None admin] PATH_INFO: `/v3/users/ada22893c13a4cc7a54346bb68b1eb71` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:13.724114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f228e-49ea-4080-b991-50d578aa03b3 None admin] RBAC: Authorizing `identity:get_user(user_id=ada22893c13a4cc7a54346bb68b1eb71)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:13.725540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f228e-49ea-4080-b991-50d578aa03b3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:13.727263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 573/1143] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:43:13 2022] GET /identity/v3/users/ada22893c13a4cc7a54346bb68b1eb71 => generated 287 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:13.733645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c3076-00f7-49f5-b6a2-871c2e6c5924 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:13.829877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c3076-00f7-49f5-b6a2-871c2e6c5924 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:13.852116 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:13.862767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16e905-4afe-45c5-a6ca-46a7a41937fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:13.863398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16e905-4afe-45c5-a6ca-46a7a41937fd None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:13.863678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16e905-4afe-45c5-a6ca-46a7a41937fd None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:13.863949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16e905-4afe-45c5-a6ca-46a7a41937fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:13.928442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16e905-4afe-45c5-a6ca-46a7a41937fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:13.929853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16e905-4afe-45c5-a6ca-46a7a41937fd None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:14.022915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 571/1144] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:43:13 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:43:14.029486 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f228e-49ea-4080-b991-50d578aa03b3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:14.062710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:14.127582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f228e-49ea-4080-b991-50d578aa03b3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:14.160585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57be92-7ebe-47b0-b3f3-c7361bab7eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:14.161200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57be92-7ebe-47b0-b3f3-c7361bab7eeb None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:14.161477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57be92-7ebe-47b0-b3f3-c7361bab7eeb None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:14.161749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57be92-7ebe-47b0-b3f3-c7361bab7eeb None admin] PATH_INFO: `/v3/projects/ef85a230b0fc441f9511711d55c656f0` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:14.162854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57be92-7ebe-47b0-b3f3-c7361bab7eeb None admin] RBAC: Authorizing `identity:get_project(project_id=ef85a230b0fc441f9511711d55c656f0)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:14.164299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57be92-7ebe-47b0-b3f3-c7361bab7eeb None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:14.166525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 574/1145] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:43:14 2022] GET /identity/v3/projects/ef85a230b0fc441f9511711d55c656f0 => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:14.172866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16e905-4afe-45c5-a6ca-46a7a41937fd None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:14.198260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:14.270724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16e905-4afe-45c5-a6ca-46a7a41937fd None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:14.303906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b02a36-9009-4125-ae7e-3b98bc385037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:14.304492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b02a36-9009-4125-ae7e-3b98bc385037 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:14.304765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b02a36-9009-4125-ae7e-3b98bc385037 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:14.305030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b02a36-9009-4125-ae7e-3b98bc385037 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:14.306804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b02a36-9009-4125-ae7e-3b98bc385037 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=ef85a230b0fc441f9511711d55c656f0, user.id=ada22893c13a4cc7a54346bb68b1eb71) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:14.307129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b02a36-9009-4125-ae7e-3b98bc385037 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:14.308537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b02a36-9009-4125-ae7e-3b98bc385037 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:14.313616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 572/1146] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:43:14 2022] GET /identity/v3/role_assignments?user.id=ada22893c13a4cc7a54346bb68b1eb71&scope.project.id=ef85a230b0fc441f9511711d55c656f0&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 260 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:16.178923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c7b55-17aa-4547-ba38-de417e28e9bd None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:16.179473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c7b55-17aa-4547-ba38-de417e28e9bd None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:16.179567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c7b55-17aa-4547-ba38-de417e28e9bd None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:16.188579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 575/1147] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:16.188713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf083e9-655d-424d-8820-1e9e9c44018e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:16.188713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf083e9-655d-424d-8820-1e9e9c44018e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:16.188713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf083e9-655d-424d-8820-1e9e9c44018e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:16.205870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-aaf083e9-655d-424d-8820-1e9e9c44018e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:16.207075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-aaf083e9-655d-424d-8820-1e9e9c44018e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:16.270804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 573/1148] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:16 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:16.538903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c7b55-17aa-4547-ba38-de417e28e9bd None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:16.636505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c7b55-17aa-4547-ba38-de417e28e9bd None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:16.638001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:16.669455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3d864-406b-49af-af05-0587eedc9872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:16.670067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3d864-406b-49af-af05-0587eedc9872 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:16.670342 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3d864-406b-49af-af05-0587eedc9872 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:16.670611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3d864-406b-49af-af05-0587eedc9872 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:16.681195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3d864-406b-49af-af05-0587eedc9872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-3ed3d864-406b-49af-af05-0587eedc9872 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:43:16.685787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:43:16.686962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:43:16.687896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 576/1149] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:43:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:43:16.693144 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf083e9-655d-424d-8820-1e9e9c44018e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:16.711250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:16.788074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf083e9-655d-424d-8820-1e9e9c44018e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:16.820417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff623a-060c-4f0d-8125-61bc315c012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:16.821008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff623a-060c-4f0d-8125-61bc315c012a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:16.821851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff623a-060c-4f0d-8125-61bc315c012a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:16.822124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff623a-060c-4f0d-8125-61bc315c012a None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:16.822862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff623a-060c-4f0d-8125-61bc315c012a None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:16.823192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff623a-060c-4f0d-8125-61bc315c012a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:16.824537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff623a-060c-4f0d-8125-61bc315c012a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:16.832195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 574/1150] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:43:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:16.838210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3d864-406b-49af-af05-0587eedc9872 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:16.934378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3d864-406b-49af-af05-0587eedc9872 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:16.967368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3dc973-303b-4f81-910d-ea0e3a5d2520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:16.967969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dc973-303b-4f81-910d-ea0e3a5d2520 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:16.968244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dc973-303b-4f81-910d-ea0e3a5d2520 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:16.968512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dc973-303b-4f81-910d-ea0e3a5d2520 None admin] PATH_INFO: `/v3/users/ada22893c13a4cc7a54346bb68b1eb71` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:16.969607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dc973-303b-4f81-910d-ea0e3a5d2520 None admin] RBAC: Authorizing `identity:get_user(user_id=ada22893c13a4cc7a54346bb68b1eb71)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:16.971012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dc973-303b-4f81-910d-ea0e3a5d2520 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:16.972560 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 577/1151] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:43:16 2022] GET /identity/v3/users/ada22893c13a4cc7a54346bb68b1eb71 => generated 287 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:16.978363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff623a-060c-4f0d-8125-61bc315c012a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:17.058976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:17.077636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff623a-060c-4f0d-8125-61bc315c012a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:17.110603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd6fe7-47d1-4fad-80ed-1deca86d7118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:17.111250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6fe7-47d1-4fad-80ed-1deca86d7118 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:17.111529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6fe7-47d1-4fad-80ed-1deca86d7118 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:17.111796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd6fe7-47d1-4fad-80ed-1deca86d7118 None admin] PATH_INFO: `/v3/projects/ef85a230b0fc441f9511711d55c656f0` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:17.112887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd6fe7-47d1-4fad-80ed-1deca86d7118 None admin] RBAC: Authorizing `identity:get_project(project_id=ef85a230b0fc441f9511711d55c656f0)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:17.114303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd6fe7-47d1-4fad-80ed-1deca86d7118 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:17.115944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 575/1152] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:43:16 2022] GET /identity/v3/projects/ef85a230b0fc441f9511711d55c656f0 => generated 312 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:17.122448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3dc973-303b-4f81-910d-ea0e3a5d2520 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:17.152522 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:17.219882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3dc973-303b-4f81-910d-ea0e3a5d2520 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:17.252822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff754fca-dd5e-47c8-a72c-919870948e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:17.253465 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff754fca-dd5e-47c8-a72c-919870948e56 None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:17.253743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff754fca-dd5e-47c8-a72c-919870948e56 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:17.254014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff754fca-dd5e-47c8-a72c-919870948e56 None admin] PATH_INFO: `/v3/projects/ef85a230b0fc441f9511711d55c656f0/users/ada22893c13a4cc7a54346bb68b1eb71/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:17.255926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff754fca-dd5e-47c8-a72c-919870948e56 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef85a230b0fc441f9511711d55c656f0, user_id=ada22893c13a4cc7a54346bb68b1eb71, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:17.257334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff754fca-dd5e-47c8-a72c-919870948e56 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:17.360306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 578/1153] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:43:17 2022] PUT /identity/v3/projects/ef85a230b0fc441f9511711d55c656f0/users/ada22893c13a4cc7a54346bb68b1eb71/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:43:19.234634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbd60e-a8e7-4d77-990b-6a18ecfff865 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:19.235122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbd60e-a8e7-4d77-990b-6a18ecfff865 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:19.235217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbd60e-a8e7-4d77-990b-6a18ecfff865 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:19.236546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 576/1154] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:19.241791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a377e-e79a-4dfd-8e63-4a8662762f0c None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:19.242161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a377e-e79a-4dfd-8e63-4a8662762f0c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:19.242341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a377e-e79a-4dfd-8e63-4a8662762f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:19.261464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2f7a377e-e79a-4dfd-8e63-4a8662762f0c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:19.262635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2f7a377e-e79a-4dfd-8e63-4a8662762f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:19.327317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 579/1155] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:19 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:19.594746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbd60e-a8e7-4d77-990b-6a18ecfff865 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:19.692869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbd60e-a8e7-4d77-990b-6a18ecfff865 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:19.725631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed2063-96fb-457b-8d08-cc2361855736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:19.726238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed2063-96fb-457b-8d08-cc2361855736 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:19.726515 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed2063-96fb-457b-8d08-cc2361855736 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:19.726783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed2063-96fb-457b-8d08-cc2361855736 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:19.736991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed2063-96fb-457b-8d08-cc2361855736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-92ed2063-96fb-457b-8d08-cc2361855736 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:43:19.741597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:43:19.742675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:43:19.744130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 577/1156] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:43:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:43:19.748602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a377e-e79a-4dfd-8e63-4a8662762f0c None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:19.768916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:19.844979 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a377e-e79a-4dfd-8e63-4a8662762f0c None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:19.878262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1fb7-f254-40c1-ac6d-c3a7dda1eddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:19.878851 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1fb7-f254-40c1-ac6d-c3a7dda1eddf None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:19.879155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1fb7-f254-40c1-ac6d-c3a7dda1eddf None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:19.879421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1fb7-f254-40c1-ac6d-c3a7dda1eddf None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:19.880067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb1fb7-f254-40c1-ac6d-c3a7dda1eddf None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:19.880359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb1fb7-f254-40c1-ac6d-c3a7dda1eddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:19.881688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb1fb7-f254-40c1-ac6d-c3a7dda1eddf None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:19.889634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 580/1157] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:43:19 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:19.895308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed2063-96fb-457b-8d08-cc2361855736 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:19.907035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:19.992683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed2063-96fb-457b-8d08-cc2361855736 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:20.025609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd8904-a513-4d62-b0d7-6803ec36cbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:20.026222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd8904-a513-4d62-b0d7-6803ec36cbd1 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:20.026501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd8904-a513-4d62-b0d7-6803ec36cbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:20.026770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd8904-a513-4d62-b0d7-6803ec36cbd1 None admin] PATH_INFO: `/v3/users/ada22893c13a4cc7a54346bb68b1eb71` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:20.027949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd8904-a513-4d62-b0d7-6803ec36cbd1 None admin] RBAC: Authorizing `identity:get_user(user_id=ada22893c13a4cc7a54346bb68b1eb71)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:20.029361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd8904-a513-4d62-b0d7-6803ec36cbd1 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:20.030914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 578/1158] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:43:19 2022] GET /identity/v3/users/ada22893c13a4cc7a54346bb68b1eb71 => generated 287 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:20.036655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1fb7-f254-40c1-ac6d-c3a7dda1eddf None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:20.117089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:20.134893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1fb7-f254-40c1-ac6d-c3a7dda1eddf None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:20.168568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e05518-b266-464f-881e-ca6bc1b2c27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:20.169162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e05518-b266-464f-881e-ca6bc1b2c27a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:20.169436 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e05518-b266-464f-881e-ca6bc1b2c27a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:20.169700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e05518-b266-464f-881e-ca6bc1b2c27a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:20.236684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e05518-b266-464f-881e-ca6bc1b2c27a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:20.238122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e05518-b266-464f-881e-ca6bc1b2c27a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:20.314986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:20.333301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 581/1159] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:43:20 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:43:20.340343 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd8904-a513-4d62-b0d7-6803ec36cbd1 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:20.414758 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:20.437079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd8904-a513-4d62-b0d7-6803ec36cbd1 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:20.470661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a8097-b061-4cba-b3bb-0ec6e57e9a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:20.471306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a8097-b061-4cba-b3bb-0ec6e57e9a5e None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:20.471584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a8097-b061-4cba-b3bb-0ec6e57e9a5e None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:20.471852 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a8097-b061-4cba-b3bb-0ec6e57e9a5e None admin] PATH_INFO: `/v3/projects/ef85a230b0fc441f9511711d55c656f0` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:20.472952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a8097-b061-4cba-b3bb-0ec6e57e9a5e None admin] RBAC: Authorizing `identity:get_project(project_id=ef85a230b0fc441f9511711d55c656f0)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:20.474364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a8097-b061-4cba-b3bb-0ec6e57e9a5e None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:20.475981 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 579/1160] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:43:20 2022] GET /identity/v3/projects/ef85a230b0fc441f9511711d55c656f0 => generated 312 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:20.482295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e05518-b266-464f-881e-ca6bc1b2c27a None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:20.578411 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e05518-b266-464f-881e-ca6bc1b2c27a None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:20.611700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553098f9-2e35-4f65-980b-c8faea1b967d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:20.612289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553098f9-2e35-4f65-980b-c8faea1b967d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:20.612570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553098f9-2e35-4f65-980b-c8faea1b967d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:20.612840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553098f9-2e35-4f65-980b-c8faea1b967d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:20.613552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553098f9-2e35-4f65-980b-c8faea1b967d None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=ef85a230b0fc441f9511711d55c656f0, user.id=ada22893c13a4cc7a54346bb68b1eb71) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:20.613858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553098f9-2e35-4f65-980b-c8faea1b967d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:20.615295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553098f9-2e35-4f65-980b-c8faea1b967d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:20.621504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 582/1161] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:43:20 2022] GET /identity/v3/role_assignments?user.id=ada22893c13a4cc7a54346bb68b1eb71&scope.project.id=ef85a230b0fc441f9511711d55c656f0&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 610 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:22.538799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82523bb3-fa71-4a33-b4d8-f1ab2fe16cf9 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:22.539324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82523bb3-fa71-4a33-b4d8-f1ab2fe16cf9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:22.539324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82523bb3-fa71-4a33-b4d8-f1ab2fe16cf9 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:22.541261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 580/1162] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:22 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:22.546425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb9493-bc8f-490a-8869-2a4a54e5e445 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:22.546820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb9493-bc8f-490a-8869-2a4a54e5e445 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:22.546971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb9493-bc8f-490a-8869-2a4a54e5e445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:22.565404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-65fb9493-bc8f-490a-8869-2a4a54e5e445 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:22.566578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-65fb9493-bc8f-490a-8869-2a4a54e5e445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:22.607344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:22.630977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 583/1163] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:22 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:22.898633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82523bb3-fa71-4a33-b4d8-f1ab2fe16cf9 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:22.997793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82523bb3-fa71-4a33-b4d8-f1ab2fe16cf9 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:23.030867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa1772-c6f7-48b7-a808-ba6bacf7f0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:23.031487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa1772-c6f7-48b7-a808-ba6bacf7f0e5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:23.031767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa1772-c6f7-48b7-a808-ba6bacf7f0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:23.032036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa1772-c6f7-48b7-a808-ba6bacf7f0e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:23.047893 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:23.100154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa1772-c6f7-48b7-a808-ba6bacf7f0e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:23.101581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa1772-c6f7-48b7-a808-ba6bacf7f0e5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:23.195288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 581/1164] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:43:22 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:43:23.201875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb9493-bc8f-490a-8869-2a4a54e5e445 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:23.297957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb9493-bc8f-490a-8869-2a4a54e5e445 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:23.331961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12217d67-60c5-41d6-b76f-ec71d60c284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:23.332571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12217d67-60c5-41d6-b76f-ec71d60c284b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:23.332846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12217d67-60c5-41d6-b76f-ec71d60c284b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:23.333108 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12217d67-60c5-41d6-b76f-ec71d60c284b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:23.340590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12217d67-60c5-41d6-b76f-ec71d60c284b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:23.342002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12217d67-60c5-41d6-b76f-ec71d60c284b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-12217d67-60c5-41d6-b76f-ec71d60c284b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:43:23.353317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:43:23.354611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:43:23.355624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 584/1165] 10.222.0.105 () {56 vars in 1122 bytes} [Wed Aug 24 04:43:23 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:43:23.360369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa1772-c6f7-48b7-a808-ba6bacf7f0e5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:23.405137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:23.458337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa1772-c6f7-48b7-a808-ba6bacf7f0e5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:23.490237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c036b-03b1-498b-9c7b-901361bb4c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:23.490835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c036b-03b1-498b-9c7b-901361bb4c8d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:23.491143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c036b-03b1-498b-9c7b-901361bb4c8d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:23.491416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c036b-03b1-498b-9c7b-901361bb4c8d None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:23.492047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c036b-03b1-498b-9c7b-901361bb4c8d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:23.492348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c036b-03b1-498b-9c7b-901361bb4c8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:23.494235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c036b-03b1-498b-9c7b-901361bb4c8d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:23.501111 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 582/1166] 10.222.0.105 () {56 vars in 1120 bytes} [Wed Aug 24 04:43:23 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:25.393573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ca5a4-ee76-487f-a9aa-54c52c2a6636 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:25.394103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ca5a4-ee76-487f-a9aa-54c52c2a6636 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:25.394103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ca5a4-ee76-487f-a9aa-54c52c2a6636 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:25.395481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 585/1167] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:25.400850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3c337-15b2-40d4-856f-225bb638863c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:25.401250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3c337-15b2-40d4-856f-225bb638863c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:25.401402 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3c337-15b2-40d4-856f-225bb638863c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:25.419907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c6f3c337-15b2-40d4-856f-225bb638863c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:25.421079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c6f3c337-15b2-40d4-856f-225bb638863c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:25.485977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 583/1168] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:25 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:25.756053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ca5a4-ee76-487f-a9aa-54c52c2a6636 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:25.792718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:25.856175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ca5a4-ee76-487f-a9aa-54c52c2a6636 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:25.888546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f180bcf-75ea-4034-94a6-345c4cd1c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:25.889159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f180bcf-75ea-4034-94a6-345c4cd1c25f None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:25.889432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f180bcf-75ea-4034-94a6-345c4cd1c25f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:25.889700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f180bcf-75ea-4034-94a6-345c4cd1c25f None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:25.890348 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f180bcf-75ea-4034-94a6-345c4cd1c25f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:25.891728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f180bcf-75ea-4034-94a6-345c4cd1c25f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:25.961139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 586/1169] 10.222.0.105 () {60 vars in 1143 bytes} [Wed Aug 24 04:43:25 2022] POST /identity/v3/domains => generated 268 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:43:27.958405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab0caf-329e-4e50-883d-e00d87418a33 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:27.958968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab0caf-329e-4e50-883d-e00d87418a33 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:27.958968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab0caf-329e-4e50-883d-e00d87418a33 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:27.960317 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 584/1170] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:27.965587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e9e50-033b-4edc-897d-92c6f6d6269f None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:27.965985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e9e50-033b-4edc-897d-92c6f6d6269f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:27.966132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e9e50-033b-4edc-897d-92c6f6d6269f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:27.983810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-302e9e50-033b-4edc-897d-92c6f6d6269f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:27.984984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-302e9e50-033b-4edc-897d-92c6f6d6269f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:28.048902 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 587/1171] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:27 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:28.317421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab0caf-329e-4e50-883d-e00d87418a33 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:28.357449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:28.414496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab0caf-329e-4e50-883d-e00d87418a33 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:28.448574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f319186-4f36-4c1e-8541-9d91a617705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:28.449185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f319186-4f36-4c1e-8541-9d91a617705f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:28.449460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f319186-4f36-4c1e-8541-9d91a617705f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:28.449730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f319186-4f36-4c1e-8541-9d91a617705f None admin] PATH_INFO: `/v3/domains/73d470256cf64ccc8958d46730dc5924` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:28.459704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f319186-4f36-4c1e-8541-9d91a617705f None admin] RBAC: Authorizing `identity:get_domain(domain_id=73d470256cf64ccc8958d46730dc5924)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:28.461121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f319186-4f36-4c1e-8541-9d91a617705f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:28.462802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 585/1172] 10.222.0.105 () {56 vars in 1188 bytes} [Wed Aug 24 04:43:28 2022] GET /identity/v3/domains/73d470256cf64ccc8958d46730dc5924 => generated 268 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:28.468640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e9e50-033b-4edc-897d-92c6f6d6269f None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:28.495120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:28.564790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e9e50-033b-4edc-897d-92c6f6d6269f None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:28.597585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5bab4c-8e01-4648-8682-95b297d488a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:28.598123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bab4c-8e01-4648-8682-95b297d488a7 None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:28.598401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bab4c-8e01-4648-8682-95b297d488a7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:28.598673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bab4c-8e01-4648-8682-95b297d488a7 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:28.599334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bab4c-8e01-4648-8682-95b297d488a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:28.600751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bab4c-8e01-4648-8682-95b297d488a7 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:28.692485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 588/1173] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:43:28 2022] POST /identity/v3/projects => generated 362 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:43:30.615449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801c028-0546-4a84-972c-c1adac853e8e None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:30.616021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801c028-0546-4a84-972c-c1adac853e8e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:30.616021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801c028-0546-4a84-972c-c1adac853e8e None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:30.617323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 586/1174] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:30.623136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c50c49-d03c-4771-a5c1-cbb5148007bb None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:30.623568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c50c49-d03c-4771-a5c1-cbb5148007bb None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:30.623698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c50c49-d03c-4771-a5c1-cbb5148007bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:30.642193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-15c50c49-d03c-4771-a5c1-cbb5148007bb None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:30.643416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-15c50c49-d03c-4771-a5c1-cbb5148007bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:30.708426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 589/1175] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:30 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:30.977307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801c028-0546-4a84-972c-c1adac853e8e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:31.072789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:31.077607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801c028-0546-4a84-972c-c1adac853e8e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:31.111667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0d697-af80-48a0-934f-92dd5f786523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:31.112284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0d697-af80-48a0-934f-92dd5f786523 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:31.112563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0d697-af80-48a0-934f-92dd5f786523 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:31.112834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0d697-af80-48a0-934f-92dd5f786523 None admin] PATH_INFO: `/v3/domains/73d470256cf64ccc8958d46730dc5924` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:31.113939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0d697-af80-48a0-934f-92dd5f786523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=73d470256cf64ccc8958d46730dc5924)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:31.115536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0d697-af80-48a0-934f-92dd5f786523 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:31.117082 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 587/1176] 10.222.0.105 () {56 vars in 1188 bytes} [Wed Aug 24 04:43:30 2022] GET /identity/v3/domains/73d470256cf64ccc8958d46730dc5924 => generated 268 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:31.123381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c50c49-d03c-4771-a5c1-cbb5148007bb None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:31.133303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:31.220530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c50c49-d03c-4771-a5c1-cbb5148007bb None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:31.253949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fa666-5f21-4184-916d-8c3cfe8e300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:31.254486 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fa666-5f21-4184-916d-8c3cfe8e300e None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:31.254763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fa666-5f21-4184-916d-8c3cfe8e300e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:31.255054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fa666-5f21-4184-916d-8c3cfe8e300e None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:31.255682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fa666-5f21-4184-916d-8c3cfe8e300e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:31.257085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fa666-5f21-4184-916d-8c3cfe8e300e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:31.338931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 590/1177] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:43:31 2022] POST /identity/v3/users => generated 312 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:43:33.278261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bac6a-a692-4c76-8e3d-44c3143d4543 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:33.278778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bac6a-a692-4c76-8e3d-44c3143d4543 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:33.278778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bac6a-a692-4c76-8e3d-44c3143d4543 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:33.280150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 588/1178] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:33.285313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f17dc-d501-4a66-9538-834693190149 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:33.285698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f17dc-d501-4a66-9538-834693190149 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:33.285863 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f17dc-d501-4a66-9538-834693190149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:33.304257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d59f17dc-d501-4a66-9538-834693190149 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:33.305441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d59f17dc-d501-4a66-9538-834693190149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:33.362972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:33.371100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 591/1179] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:33 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:33.639814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bac6a-a692-4c76-8e3d-44c3143d4543 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:33.737818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bac6a-a692-4c76-8e3d-44c3143d4543 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:33.770277 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d564b-21e3-4f79-8c9e-766d508513bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:33.770880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d564b-21e3-4f79-8c9e-766d508513bf None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:33.771184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d564b-21e3-4f79-8c9e-766d508513bf None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:33.771454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d564b-21e3-4f79-8c9e-766d508513bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:33.781989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d564b-21e3-4f79-8c9e-766d508513bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-cf1d564b-21e3-4f79-8c9e-766d508513bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:43:33.786554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:43:33.787860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:43:33.788975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 589/1180] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:43:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:43:33.793782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f17dc-d501-4a66-9538-834693190149 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:33.888330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f17dc-d501-4a66-9538-834693190149 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:33.920812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09d044-0ed7-4bfc-8b84-4f00e79a8547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:33.921347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09d044-0ed7-4bfc-8b84-4f00e79a8547 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:33.921625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09d044-0ed7-4bfc-8b84-4f00e79a8547 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:33.921895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09d044-0ed7-4bfc-8b84-4f00e79a8547 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:33.922548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09d044-0ed7-4bfc-8b84-4f00e79a8547 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:33.922845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09d044-0ed7-4bfc-8b84-4f00e79a8547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:33.924230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed09d044-0ed7-4bfc-8b84-4f00e79a8547 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:33.932574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 592/1181] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:43:33 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:33.938187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d564b-21e3-4f79-8c9e-766d508513bf None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:33.965816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:34.036713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d564b-21e3-4f79-8c9e-766d508513bf None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:34.069471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc452656-37ad-4fbf-8eb5-1d97fd6e245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:34.070080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc452656-37ad-4fbf-8eb5-1d97fd6e245a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:34.070356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc452656-37ad-4fbf-8eb5-1d97fd6e245a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:34.070623 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc452656-37ad-4fbf-8eb5-1d97fd6e245a None admin] PATH_INFO: `/v3/users/b80e2548416a44f8ab8b666316f9069c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:34.083768 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc452656-37ad-4fbf-8eb5-1d97fd6e245a None admin] RBAC: Authorizing `identity:get_user(user_id=b80e2548416a44f8ab8b666316f9069c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:34.085186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc452656-37ad-4fbf-8eb5-1d97fd6e245a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:34.086753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 590/1182] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:43:33 2022] GET /identity/v3/users/b80e2548416a44f8ab8b666316f9069c => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:34.092702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09d044-0ed7-4bfc-8b84-4f00e79a8547 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:34.147556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:34.190610 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed09d044-0ed7-4bfc-8b84-4f00e79a8547 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:34.224467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc90e-319e-42d4-990e-de2dc5f5dade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:34.225072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc90e-319e-42d4-990e-de2dc5f5dade None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:34.225352 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc90e-319e-42d4-990e-de2dc5f5dade None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:34.225625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acc90e-319e-42d4-990e-de2dc5f5dade None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:34.290808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acc90e-319e-42d4-990e-de2dc5f5dade None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:34.292882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acc90e-319e-42d4-990e-de2dc5f5dade None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:34.345467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:34.388817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 593/1183] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:43:34 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:43:34.394783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc452656-37ad-4fbf-8eb5-1d97fd6e245a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:34.475472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:34.490868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc452656-37ad-4fbf-8eb5-1d97fd6e245a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:34.523622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c22ab1-0563-4cb9-9a9c-e23973050302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:34.524234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c22ab1-0563-4cb9-9a9c-e23973050302 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:34.524509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c22ab1-0563-4cb9-9a9c-e23973050302 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:34.524774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c22ab1-0563-4cb9-9a9c-e23973050302 None admin] PATH_INFO: `/v3/projects/8da35c55e49d470c80dad7c0919bf502` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:34.526070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c22ab1-0563-4cb9-9a9c-e23973050302 None admin] RBAC: Authorizing `identity:get_project(project_id=8da35c55e49d470c80dad7c0919bf502)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:34.527521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c22ab1-0563-4cb9-9a9c-e23973050302 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:34.529107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 591/1184] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:43:34 2022] GET /identity/v3/projects/8da35c55e49d470c80dad7c0919bf502 => generated 362 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:34.535054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc90e-319e-42d4-990e-de2dc5f5dade None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:34.631235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acc90e-319e-42d4-990e-de2dc5f5dade None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:34.663938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e24f4c-1a70-47fb-a7ba-964d1c4aca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:34.664523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e24f4c-1a70-47fb-a7ba-964d1c4aca2d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:34.664794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e24f4c-1a70-47fb-a7ba-964d1c4aca2d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:34.665059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e24f4c-1a70-47fb-a7ba-964d1c4aca2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:34.665766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e24f4c-1a70-47fb-a7ba-964d1c4aca2d None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=8da35c55e49d470c80dad7c0919bf502, user.id=b80e2548416a44f8ab8b666316f9069c) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:34.666705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e24f4c-1a70-47fb-a7ba-964d1c4aca2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:34.668322 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e24f4c-1a70-47fb-a7ba-964d1c4aca2d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:34.673395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 594/1185] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:43:34 2022] GET /identity/v3/role_assignments?user.id=b80e2548416a44f8ab8b666316f9069c&scope.project.id=8da35c55e49d470c80dad7c0919bf502&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 260 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:36.545698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4381863-0fab-4056-8e88-277aa77ebe1b None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:36.546222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4381863-0fab-4056-8e88-277aa77ebe1b None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:36.546222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4381863-0fab-4056-8e88-277aa77ebe1b None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:36.547586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 592/1186] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:36.552803 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0facc-233b-47c6-b744-293d8b25240e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:36.553190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0facc-233b-47c6-b744-293d8b25240e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:36.553352 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0facc-233b-47c6-b744-293d8b25240e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:36.571899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-87e0facc-233b-47c6-b744-293d8b25240e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:36.573773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-87e0facc-233b-47c6-b744-293d8b25240e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:36.587249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:36.638557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 595/1187] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:36 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:36.907100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4381863-0fab-4056-8e88-277aa77ebe1b None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:37.005443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4381863-0fab-4056-8e88-277aa77ebe1b None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:37.037874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250365ce-fd95-4ec3-a522-d5793f979b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:37.039417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250365ce-fd95-4ec3-a522-d5793f979b13 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:37.039417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250365ce-fd95-4ec3-a522-d5793f979b13 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:37.039417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250365ce-fd95-4ec3-a522-d5793f979b13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:37.048900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250365ce-fd95-4ec3-a522-d5793f979b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-250365ce-fd95-4ec3-a522-d5793f979b13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:43:37.053441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:43:37.054556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:43:37.055453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 593/1188] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:43:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:43:37.061292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0facc-233b-47c6-b744-293d8b25240e None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:37.158236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0facc-233b-47c6-b744-293d8b25240e None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:37.191817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d61ec-47bb-4b37-a248-a5db934f5169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:37.192420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d61ec-47bb-4b37-a248-a5db934f5169 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:37.192694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d61ec-47bb-4b37-a248-a5db934f5169 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:37.192958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d61ec-47bb-4b37-a248-a5db934f5169 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:37.193607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d61ec-47bb-4b37-a248-a5db934f5169 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:37.193900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d61ec-47bb-4b37-a248-a5db934f5169 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:37.195277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546d61ec-47bb-4b37-a248-a5db934f5169 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:37.204089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 596/1189] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:43:37 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:37.209941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250365ce-fd95-4ec3-a522-d5793f979b13 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:37.224968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:37.307956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250365ce-fd95-4ec3-a522-d5793f979b13 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:37.340366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d45fe8-fdf9-4d34-b183-b1435da2c12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:37.340978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45fe8-fdf9-4d34-b183-b1435da2c12d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:37.341254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45fe8-fdf9-4d34-b183-b1435da2c12d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:37.341524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45fe8-fdf9-4d34-b183-b1435da2c12d None admin] PATH_INFO: `/v3/users/b80e2548416a44f8ab8b666316f9069c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:37.342636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d45fe8-fdf9-4d34-b183-b1435da2c12d None admin] RBAC: Authorizing `identity:get_user(user_id=b80e2548416a44f8ab8b666316f9069c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:37.344084 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d45fe8-fdf9-4d34-b183-b1435da2c12d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:37.345635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 594/1190] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:43:37 2022] GET /identity/v3/users/b80e2548416a44f8ab8b666316f9069c => generated 312 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:37.351462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d61ec-47bb-4b37-a248-a5db934f5169 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:37.366929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:37.451104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d61ec-47bb-4b37-a248-a5db934f5169 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:37.483966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92db0982-18fe-4661-8225-1b0280ea8374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:37.484573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db0982-18fe-4661-8225-1b0280ea8374 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:37.484846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db0982-18fe-4661-8225-1b0280ea8374 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:37.485110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92db0982-18fe-4661-8225-1b0280ea8374 None admin] PATH_INFO: `/v3/projects/8da35c55e49d470c80dad7c0919bf502` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:37.486206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db0982-18fe-4661-8225-1b0280ea8374 None admin] RBAC: Authorizing `identity:get_project(project_id=8da35c55e49d470c80dad7c0919bf502)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:37.487644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92db0982-18fe-4661-8225-1b0280ea8374 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:37.489229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 597/1191] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:43:37 2022] GET /identity/v3/projects/8da35c55e49d470c80dad7c0919bf502 => generated 362 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:37.495986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d45fe8-fdf9-4d34-b183-b1435da2c12d None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:37.572533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:37.590432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d45fe8-fdf9-4d34-b183-b1435da2c12d None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:37.623656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823b40a-02fb-4568-ba40-54ca49a4dde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:37.624300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823b40a-02fb-4568-ba40-54ca49a4dde3 None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:37.624582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823b40a-02fb-4568-ba40-54ca49a4dde3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:37.624855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823b40a-02fb-4568-ba40-54ca49a4dde3 None admin] PATH_INFO: `/v3/projects/8da35c55e49d470c80dad7c0919bf502/users/b80e2548416a44f8ab8b666316f9069c/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:37.626736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823b40a-02fb-4568-ba40-54ca49a4dde3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8da35c55e49d470c80dad7c0919bf502, user_id=b80e2548416a44f8ab8b666316f9069c, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:37.628184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823b40a-02fb-4568-ba40-54ca49a4dde3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:37.704889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 595/1192] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:43:37 2022] PUT /identity/v3/projects/8da35c55e49d470c80dad7c0919bf502/users/b80e2548416a44f8ab8b666316f9069c/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:43:39.591771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbef51-3e63-484c-8cbc-f3b38c3ad999 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:39.592306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbef51-3e63-484c-8cbc-f3b38c3ad999 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:39.592306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbef51-3e63-484c-8cbc-f3b38c3ad999 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:39.593684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 598/1193] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:43:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:43:39.598776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdbcf4-5edd-42cf-bd46-1ee2cbdf8e74 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:39.599218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdbcf4-5edd-42cf-bd46-1ee2cbdf8e74 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:39.599343 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cdbcf4-5edd-42cf-bd46-1ee2cbdf8e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:39.616976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-95cdbcf4-5edd-42cf-bd46-1ee2cbdf8e74 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:43:39.618151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-95cdbcf4-5edd-42cf-bd46-1ee2cbdf8e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:43:39.681716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 596/1194] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:43:39 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:43:39.951463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbef51-3e63-484c-8cbc-f3b38c3ad999 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:40.030534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:40.049189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbef51-3e63-484c-8cbc-f3b38c3ad999 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:40.083751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772c7e6-4603-4c88-8d0e-1959a9575b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:40.084367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772c7e6-4603-4c88-8d0e-1959a9575b3a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:40.084645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772c7e6-4603-4c88-8d0e-1959a9575b3a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:40.084918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772c7e6-4603-4c88-8d0e-1959a9575b3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:40.095115 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772c7e6-4603-4c88-8d0e-1959a9575b3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-f772c7e6-4603-4c88-8d0e-1959a9575b3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:43:40.099798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:43:40.100909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:43:40.102105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 599/1195] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:43:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:43:40.107247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdbcf4-5edd-42cf-bd46-1ee2cbdf8e74 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:40.174699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:40.204503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cdbcf4-5edd-42cf-bd46-1ee2cbdf8e74 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:40.238883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319fec3-f6de-419f-ae9c-5b058600a8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:40.239509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319fec3-f6de-419f-ae9c-5b058600a8a5 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:40.239784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319fec3-f6de-419f-ae9c-5b058600a8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:40.240051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319fec3-f6de-419f-ae9c-5b058600a8a5 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:40.240700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319fec3-f6de-419f-ae9c-5b058600a8a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:40.240999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319fec3-f6de-419f-ae9c-5b058600a8a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:40.242348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319fec3-f6de-419f-ae9c-5b058600a8a5 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:40.250187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 597/1196] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:43:40 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:40.255952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772c7e6-4603-4c88-8d0e-1959a9575b3a None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:40.354226 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772c7e6-4603-4c88-8d0e-1959a9575b3a None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:40.387130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72d24c-1e96-4a0a-bd8c-47da34c99f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:40.387738 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72d24c-1e96-4a0a-bd8c-47da34c99f5f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:40.388018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72d24c-1e96-4a0a-bd8c-47da34c99f5f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:40.388291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72d24c-1e96-4a0a-bd8c-47da34c99f5f None admin] PATH_INFO: `/v3/users/b80e2548416a44f8ab8b666316f9069c` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:40.389405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72d24c-1e96-4a0a-bd8c-47da34c99f5f None admin] RBAC: Authorizing `identity:get_user(user_id=b80e2548416a44f8ab8b666316f9069c)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:40.390824 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72d24c-1e96-4a0a-bd8c-47da34c99f5f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:40.392413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 600/1197] 10.222.0.105 () {56 vars in 1182 bytes} [Wed Aug 24 04:43:40 2022] GET /identity/v3/users/b80e2548416a44f8ab8b666316f9069c => generated 312 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:40.398213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319fec3-f6de-419f-ae9c-5b058600a8a5 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:40.496449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319fec3-f6de-419f-ae9c-5b058600a8a5 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:40.523351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:40.527957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d844-34d9-41ef-9585-4a516e04e73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:40.528545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d844-34d9-41ef-9585-4a516e04e73c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:40.528820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d844-34d9-41ef-9585-4a516e04e73c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:40.529087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d844-34d9-41ef-9585-4a516e04e73c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:40.594318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767d844-34d9-41ef-9585-4a516e04e73c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:40.595788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767d844-34d9-41ef-9585-4a516e04e73c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:40.691895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 598/1198] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:43:40 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:43:40.697998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72d24c-1e96-4a0a-bd8c-47da34c99f5f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:40.709614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:40.796946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72d24c-1e96-4a0a-bd8c-47da34c99f5f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:40.829271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af233c74-5923-40db-94b0-e66ec2d47cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:40.829883 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af233c74-5923-40db-94b0-e66ec2d47cd6 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:40.830166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af233c74-5923-40db-94b0-e66ec2d47cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:40.830447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af233c74-5923-40db-94b0-e66ec2d47cd6 None admin] PATH_INFO: `/v3/projects/8da35c55e49d470c80dad7c0919bf502` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:40.831585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af233c74-5923-40db-94b0-e66ec2d47cd6 None admin] RBAC: Authorizing `identity:get_project(project_id=8da35c55e49d470c80dad7c0919bf502)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:40.833013 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af233c74-5923-40db-94b0-e66ec2d47cd6 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:40.834625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 601/1199] 10.222.0.105 () {56 vars in 1191 bytes} [Wed Aug 24 04:43:40 2022] GET /identity/v3/projects/8da35c55e49d470c80dad7c0919bf502 => generated 362 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:40.841888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d844-34d9-41ef-9585-4a516e04e73c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:43:40.871776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:40.937943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d844-34d9-41ef-9585-4a516e04e73c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:43:40.971179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0751bce-485a-49b2-a87b-fafa898459c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:43:40.971771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0751bce-485a-49b2-a87b-fafa898459c7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:43:40.972048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0751bce-485a-49b2-a87b-fafa898459c7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:43:40.972320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0751bce-485a-49b2-a87b-fafa898459c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:43:40.973033 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0751bce-485a-49b2-a87b-fafa898459c7 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=8da35c55e49d470c80dad7c0919bf502, user.id=b80e2548416a44f8ab8b666316f9069c) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:43:40.973336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0751bce-485a-49b2-a87b-fafa898459c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:43:40.974744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0751bce-485a-49b2-a87b-fafa898459c7 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:43:40.980941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 599/1200] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:43:40 2022] GET /identity/v3/role_assignments?user.id=b80e2548416a44f8ab8b666316f9069c&scope.project.id=8da35c55e49d470c80dad7c0919bf502&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 610 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:43:50.825158 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=62 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:43:50.977693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=52 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:52.453970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a733b-c747-4f5a-9702-f6ba72b58668 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:52.454506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a733b-c747-4f5a-9702-f6ba72b58668 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:52.454506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a733b-c747-4f5a-9702-f6ba72b58668 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:52.455877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 602/1201] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:44:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:44:52.461595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf468eb-b360-4424-86e3-411e2b641c3f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:52.462040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf468eb-b360-4424-86e3-411e2b641c3f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:52.462146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf468eb-b360-4424-86e3-411e2b641c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:52.480533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9cf468eb-b360-4424-86e3-411e2b641c3f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:44:52.487247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9cf468eb-b360-4424-86e3-411e2b641c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:44:52.545364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 600/1202] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:44:52 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 04:44:52.820156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a733b-c747-4f5a-9702-f6ba72b58668 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:52.923269 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a733b-c747-4f5a-9702-f6ba72b58668 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:52.957391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c855d-f158-4940-b0ca-997eada4b2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:52.958038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c855d-f158-4940-b0ca-997eada4b2d3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:52.958316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c855d-f158-4940-b0ca-997eada4b2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:52.958582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c855d-f158-4940-b0ca-997eada4b2d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:52.969420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c855d-f158-4940-b0ca-997eada4b2d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:52.970962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c855d-f158-4940-b0ca-997eada4b2d3 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-494c855d-f158-4940-b0ca-997eada4b2d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:44:52.985353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:44:52.986557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:44:52.987938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:44:52.987938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:44:52.987938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:44:52.987938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 603/1203] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:44:52.992558 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf468eb-b360-4424-86e3-411e2b641c3f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:53.089200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf468eb-b360-4424-86e3-411e2b641c3f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:53.116729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:53.122541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe8cb9-71e7-455d-98cd-def5a9ace9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:53.123172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8cb9-71e7-455d-98cd-def5a9ace9fc None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:53.123444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8cb9-71e7-455d-98cd-def5a9ace9fc None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:53.123711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe8cb9-71e7-455d-98cd-def5a9ace9fc None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:53.124335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe8cb9-71e7-455d-98cd-def5a9ace9fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:44:53.124630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe8cb9-71e7-455d-98cd-def5a9ace9fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:53.125981 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe8cb9-71e7-455d-98cd-def5a9ace9fc None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:53.137503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 601/1204] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:44:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:44:53.143766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c855d-f158-4940-b0ca-997eada4b2d3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:53.200128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:53.246213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c855d-f158-4940-b0ca-997eada4b2d3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:53.279072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e707561-aad4-4be8-96e9-4285f208869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:53.279672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e707561-aad4-4be8-96e9-4285f208869f None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:53.279947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e707561-aad4-4be8-96e9-4285f208869f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:53.280244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e707561-aad4-4be8-96e9-4285f208869f None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:53.280860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e707561-aad4-4be8-96e9-4285f208869f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:53.282269 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e707561-aad4-4be8-96e9-4285f208869f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:53.373359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 604/1205] 10.222.0.105 () {60 vars in 1137 bytes} [Wed Aug 24 04:44:53 2022] POST /identity/v3/users => generated 252 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:44:55.301922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e017647-d706-4a4f-b2e7-a3d3ac2d2991 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:55.302438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e017647-d706-4a4f-b2e7-a3d3ac2d2991 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:55.302557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e017647-d706-4a4f-b2e7-a3d3ac2d2991 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:55.304707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 602/1206] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:44:55 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:44:55.309983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdfa0b-6fc4-4d02-b917-c7e1c8f0dee4 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:55.310375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdfa0b-6fc4-4d02-b917-c7e1c8f0dee4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:55.310563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdfa0b-6fc4-4d02-b917-c7e1c8f0dee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:55.328777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-becdfa0b-6fc4-4d02-b917-c7e1c8f0dee4 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:44:55.329969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-becdfa0b-6fc4-4d02-b917-c7e1c8f0dee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:44:55.395892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 605/1207] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:44:55 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:44:55.664157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e017647-d706-4a4f-b2e7-a3d3ac2d2991 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:55.766563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e017647-d706-4a4f-b2e7-a3d3ac2d2991 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:55.803316 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846985bb-bf08-4c39-b7fc-c375f9865a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:55.805089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846985bb-bf08-4c39-b7fc-c375f9865a90 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:55.805361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846985bb-bf08-4c39-b7fc-c375f9865a90 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:55.805624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846985bb-bf08-4c39-b7fc-c375f9865a90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:55.815284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846985bb-bf08-4c39-b7fc-c375f9865a90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-846985bb-bf08-4c39-b7fc-c375f9865a90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:44:55.820375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:44:55.823217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:44:55.824203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 603/1208] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:44:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:44:55.827713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdfa0b-6fc4-4d02-b917-c7e1c8f0dee4 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:55.867431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:55.926346 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdfa0b-6fc4-4d02-b917-c7e1c8f0dee4 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:55.960137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c601ba-16de-4625-9e7a-744a9fc1afd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:55.960739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c601ba-16de-4625-9e7a-744a9fc1afd4 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:55.961021 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c601ba-16de-4625-9e7a-744a9fc1afd4 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:55.961292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c601ba-16de-4625-9e7a-744a9fc1afd4 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:55.961949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c601ba-16de-4625-9e7a-744a9fc1afd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:44:55.962249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c601ba-16de-4625-9e7a-744a9fc1afd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:55.963663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c601ba-16de-4625-9e7a-744a9fc1afd4 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:55.971346 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 606/1209] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:44:55 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:44:55.977842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846985bb-bf08-4c39-b7fc-c375f9865a90 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:56.028771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:56.076838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846985bb-bf08-4c39-b7fc-c375f9865a90 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:56.109381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d1eb4-a217-46ea-a286-aa1aca61b583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:56.110001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d1eb4-a217-46ea-a286-aa1aca61b583 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:56.110273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d1eb4-a217-46ea-a286-aa1aca61b583 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:56.110539 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d1eb4-a217-46ea-a286-aa1aca61b583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:56.120118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d1eb4-a217-46ea-a286-aa1aca61b583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:56.121525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d1eb4-a217-46ea-a286-aa1aca61b583 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-7f2d1eb4-a217-46ea-a286-aa1aca61b583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:44:56.135269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:44:56.138897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:44:56.140050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:44:56.140050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:44:56.140050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:44:56.140050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 604/1210] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:44:56.142340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c601ba-16de-4625-9e7a-744a9fc1afd4 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:56.241195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:56.244240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c601ba-16de-4625-9e7a-744a9fc1afd4 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:56.276865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee371c9-5fff-46a0-bbd7-5a7622f8033c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:56.277465 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee371c9-5fff-46a0-bbd7-5a7622f8033c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:56.277751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee371c9-5fff-46a0-bbd7-5a7622f8033c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:56.278024 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee371c9-5fff-46a0-bbd7-5a7622f8033c None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:56.278658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee371c9-5fff-46a0-bbd7-5a7622f8033c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:44:56.278956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee371c9-5fff-46a0-bbd7-5a7622f8033c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:56.280374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee371c9-5fff-46a0-bbd7-5a7622f8033c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:56.292206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 607/1211] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:44:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:44:56.297968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d1eb4-a217-46ea-a286-aa1aca61b583 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:56.396001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d1eb4-a217-46ea-a286-aa1aca61b583 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:56.398037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:56.430998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa9f3e-e655-4009-ab2f-6e648c82c222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:56.431654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa9f3e-e655-4009-ab2f-6e648c82c222 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:56.431932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa9f3e-e655-4009-ab2f-6e648c82c222 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:56.432200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa9f3e-e655-4009-ab2f-6e648c82c222 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:56.439043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa9f3e-e655-4009-ab2f-6e648c82c222 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:56.444218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa9f3e-e655-4009-ab2f-6e648c82c222 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-15fa9f3e-e655-4009-ab2f-6e648c82c222 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:44:56.452440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:44:56.453856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:44:56.455178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 605/1212] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:44:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:44:56.459737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee371c9-5fff-46a0-bbd7-5a7622f8033c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:56.558899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee371c9-5fff-46a0-bbd7-5a7622f8033c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:56.593485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68640-d69a-4e45-a0fd-39bc950e099f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:56.594176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68640-d69a-4e45-a0fd-39bc950e099f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:56.594464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68640-d69a-4e45-a0fd-39bc950e099f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:56.594783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d68640-d69a-4e45-a0fd-39bc950e099f None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:56.595646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d68640-d69a-4e45-a0fd-39bc950e099f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:44:56.595937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d68640-d69a-4e45-a0fd-39bc950e099f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:56.597523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d68640-d69a-4e45-a0fd-39bc950e099f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:56.617881 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 608/1213] 10.222.0.105 () {56 vars in 1148 bytes} [Wed Aug 24 04:44:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:44:56.623640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa9f3e-e655-4009-ab2f-6e648c82c222 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:56.724229 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa9f3e-e655-4009-ab2f-6e648c82c222 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:56.759653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ade5d-fd8a-4649-b852-80a5194a4f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:56.760257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ade5d-fd8a-4649-b852-80a5194a4f26 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:56.760533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ade5d-fd8a-4649-b852-80a5194a4f26 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:56.760803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ade5d-fd8a-4649-b852-80a5194a4f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:56.790181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:56.828067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ade5d-fd8a-4649-b852-80a5194a4f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:56.829588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ade5d-fd8a-4649-b852-80a5194a4f26 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:56.925468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 606/1214] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:44:56 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:44:56.932027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68640-d69a-4e45-a0fd-39bc950e099f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:56.951169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:57.034163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d68640-d69a-4e45-a0fd-39bc950e099f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:57.068217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87fe63-5ed8-4b13-b13c-86b7458a167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:57.068840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87fe63-5ed8-4b13-b13c-86b7458a167d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:57.069113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87fe63-5ed8-4b13-b13c-86b7458a167d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:57.069381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87fe63-5ed8-4b13-b13c-86b7458a167d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:57.080298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87fe63-5ed8-4b13-b13c-86b7458a167d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:57.081719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87fe63-5ed8-4b13-b13c-86b7458a167d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-3a87fe63-5ed8-4b13-b13c-86b7458a167d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:44:57.095825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:44:57.096921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:44:57.098019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:44:57.098019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:44:57.098019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:44:57.098019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 609/1215] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:44:57.102865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ade5d-fd8a-4649-b852-80a5194a4f26 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:57.163700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:57.204930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ade5d-fd8a-4649-b852-80a5194a4f26 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:57.238520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66234e8d-7c9f-4ce4-acac-a4e2cc0b2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:57.239162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66234e8d-7c9f-4ce4-acac-a4e2cc0b2e86 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:57.239443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66234e8d-7c9f-4ce4-acac-a4e2cc0b2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:57.239713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66234e8d-7c9f-4ce4-acac-a4e2cc0b2e86 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:57.240339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66234e8d-7c9f-4ce4-acac-a4e2cc0b2e86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:44:57.240638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66234e8d-7c9f-4ce4-acac-a4e2cc0b2e86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:57.242612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66234e8d-7c9f-4ce4-acac-a4e2cc0b2e86 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:57.254183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 607/1216] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:44:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:44:57.260023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87fe63-5ed8-4b13-b13c-86b7458a167d None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:57.329278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:57.359613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87fe63-5ed8-4b13-b13c-86b7458a167d None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:57.393190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedab8a5-23a3-4ec3-aa15-3287e8707b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:57.394095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab8a5-23a3-4ec3-aa15-3287e8707b0f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:57.394373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab8a5-23a3-4ec3-aa15-3287e8707b0f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:57.394641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedab8a5-23a3-4ec3-aa15-3287e8707b0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:57.401748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedab8a5-23a3-4ec3-aa15-3287e8707b0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:57.403219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedab8a5-23a3-4ec3-aa15-3287e8707b0f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-bedab8a5-23a3-4ec3-aa15-3287e8707b0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:44:57.414911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:44:57.416016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:44:57.416962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:44:57.416962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:44:57.416962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:44:57.416962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:44:57.416962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:44:57.416962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:44:57.416962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:44:57.416962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:44:57.416962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 610/1217] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:44:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:44:57.421902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66234e8d-7c9f-4ce4-acac-a4e2cc0b2e86 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:57.519985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66234e8d-7c9f-4ce4-acac-a4e2cc0b2e86 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:57.542580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:57.553657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23838d54-57ca-4ed9-bbcb-4d39c2d5dcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:57.554809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23838d54-57ca-4ed9-bbcb-4d39c2d5dcdd None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:57.555118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23838d54-57ca-4ed9-bbcb-4d39c2d5dcdd None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:57.555379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23838d54-57ca-4ed9-bbcb-4d39c2d5dcdd None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:57.556049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23838d54-57ca-4ed9-bbcb-4d39c2d5dcdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:44:57.556344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23838d54-57ca-4ed9-bbcb-4d39c2d5dcdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:57.557754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23838d54-57ca-4ed9-bbcb-4d39c2d5dcdd None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:57.568888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 608/1218] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:44:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:44:57.575166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedab8a5-23a3-4ec3-aa15-3287e8707b0f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:44:57.676528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedab8a5-23a3-4ec3-aa15-3287e8707b0f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:44:57.704334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:44:57.709973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0dcbf-f98b-41fd-a9ce-fc02bc3d22fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:44:57.711130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0dcbf-f98b-41fd-a9ce-fc02bc3d22fd None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:57.711415 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0dcbf-f98b-41fd-a9ce-fc02bc3d22fd None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:57.711687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0dcbf-f98b-41fd-a9ce-fc02bc3d22fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:57.712431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0dcbf-f98b-41fd-a9ce-fc02bc3d22fd None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=ad8825c294534a7c88877b48b5f8f07f) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:44:57.712736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0dcbf-f98b-41fd-a9ce-fc02bc3d22fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:44:57.714156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb0dcbf-f98b-41fd-a9ce-fc02bc3d22fd None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:44:57.719359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 611/1219] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:44:57 2022] GET /identity/v3/role_assignments?user.id=ad8825c294534a7c88877b48b5f8f07f&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 260 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:44:59.608905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8fb79-be4d-447a-855c-928129f13134 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:59.609451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8fb79-be4d-447a-855c-928129f13134 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:59.609451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8fb79-be4d-447a-855c-928129f13134 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:59.610791 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 609/1220] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:44:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:44:59.616218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ca64f-d8ac-4194-8609-b26ef8e62a9b None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:44:59.616623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ca64f-d8ac-4194-8609-b26ef8e62a9b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:44:59.616905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ca64f-d8ac-4194-8609-b26ef8e62a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:44:59.636170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c04ca64f-d8ac-4194-8609-b26ef8e62a9b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:44:59.637413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c04ca64f-d8ac-4194-8609-b26ef8e62a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:44:59.703614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 612/1221] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:44:59 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:44:59.977735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8fb79-be4d-447a-855c-928129f13134 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:00.078742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8fb79-be4d-447a-855c-928129f13134 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:00.111346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8bed1-22b2-4590-bf87-4c226c11a223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:00.111964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8bed1-22b2-4590-bf87-4c226c11a223 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:00.112245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8bed1-22b2-4590-bf87-4c226c11a223 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:00.112517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd8bed1-22b2-4590-bf87-4c226c11a223 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:00.122841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd8bed1-22b2-4590-bf87-4c226c11a223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-2dd8bed1-22b2-4590-bf87-4c226c11a223 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:45:00.127706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:00.128841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:00.129961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 610/1222] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:44:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:00.135456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ca64f-d8ac-4194-8609-b26ef8e62a9b None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:00.233056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ca64f-d8ac-4194-8609-b26ef8e62a9b None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:00.256281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:00.266955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b6e9f-fd5a-4070-8d02-5da597382f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:00.267586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b6e9f-fd5a-4070-8d02-5da597382f14 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:00.267866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b6e9f-fd5a-4070-8d02-5da597382f14 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:00.268169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b6e9f-fd5a-4070-8d02-5da597382f14 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:00.268819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b6e9f-fd5a-4070-8d02-5da597382f14 None admin] RBAC: Adding query filter params (name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:00.269119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b6e9f-fd5a-4070-8d02-5da597382f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:00.270479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b6e9f-fd5a-4070-8d02-5da597382f14 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:00.278480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 613/1223] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:45:00 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:00.284277 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8bed1-22b2-4590-bf87-4c226c11a223 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:00.327626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:00.382861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd8bed1-22b2-4590-bf87-4c226c11a223 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:00.416660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc766f8-66da-4209-8e27-f0d6c44bd3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:00.417277 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc766f8-66da-4209-8e27-f0d6c44bd3be None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:00.417555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc766f8-66da-4209-8e27-f0d6c44bd3be None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:00.417822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc766f8-66da-4209-8e27-f0d6c44bd3be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:00.427614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc766f8-66da-4209-8e27-f0d6c44bd3be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:00.429094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc766f8-66da-4209-8e27-f0d6c44bd3be None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-2cc766f8-66da-4209-8e27-f0d6c44bd3be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:45:00.444307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:00.445928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:45:00.447162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:45:00.447162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:00.447162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:00.447162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 611/1224] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:00.451770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b6e9f-fd5a-4070-8d02-5da597382f14 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:00.549518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b6e9f-fd5a-4070-8d02-5da597382f14 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:00.582365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b95a5-016e-4bbb-a6da-77e026e8d685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:00.582960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b95a5-016e-4bbb-a6da-77e026e8d685 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:00.583264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b95a5-016e-4bbb-a6da-77e026e8d685 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:00.583532 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b95a5-016e-4bbb-a6da-77e026e8d685 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:00.584163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b95a5-016e-4bbb-a6da-77e026e8d685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:00.584460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b95a5-016e-4bbb-a6da-77e026e8d685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:00.586315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b95a5-016e-4bbb-a6da-77e026e8d685 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:00.597625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 614/1225] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:45:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:00.604164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc766f8-66da-4209-8e27-f0d6c44bd3be None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:00.697387 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:00.704050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc766f8-66da-4209-8e27-f0d6c44bd3be None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:00.738185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0bb3-1b9e-4b62-96a0-1c5ca19e0b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:00.738987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0bb3-1b9e-4b62-96a0-1c5ca19e0b45 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:00.739325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0bb3-1b9e-4b62-96a0-1c5ca19e0b45 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:00.739594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0bb3-1b9e-4b62-96a0-1c5ca19e0b45 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:00.746483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f0bb3-1b9e-4b62-96a0-1c5ca19e0b45 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:00.750387 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f0bb3-1b9e-4b62-96a0-1c5ca19e0b45 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-1e4f0bb3-1b9e-4b62-96a0-1c5ca19e0b45 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:45:00.759251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:00.760460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:00.761715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 612/1226] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:00.766493 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b95a5-016e-4bbb-a6da-77e026e8d685 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:00.814258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:00.867608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b95a5-016e-4bbb-a6da-77e026e8d685 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:00.900249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2ff77-a115-4916-bb13-0f73ff3c98f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:00.900845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2ff77-a115-4916-bb13-0f73ff3c98f8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:00.901121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2ff77-a115-4916-bb13-0f73ff3c98f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:00.901381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2ff77-a115-4916-bb13-0f73ff3c98f8 None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:00.902099 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2ff77-a115-4916-bb13-0f73ff3c98f8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:00.902397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2ff77-a115-4916-bb13-0f73ff3c98f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:00.903827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2ff77-a115-4916-bb13-0f73ff3c98f8 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:00.921219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 615/1227] 10.222.0.105 () {56 vars in 1148 bytes} [Wed Aug 24 04:45:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:00.927092 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0bb3-1b9e-4b62-96a0-1c5ca19e0b45 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:01.026226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0bb3-1b9e-4b62-96a0-1c5ca19e0b45 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:01.059011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be32ffc-cac5-4797-b7ce-c3e89b6090ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:01.059694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be32ffc-cac5-4797-b7ce-c3e89b6090ac None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:01.059975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be32ffc-cac5-4797-b7ce-c3e89b6090ac None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:01.060249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be32ffc-cac5-4797-b7ce-c3e89b6090ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:01.070790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be32ffc-cac5-4797-b7ce-c3e89b6090ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:01.073322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be32ffc-cac5-4797-b7ce-c3e89b6090ac None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-6be32ffc-cac5-4797-b7ce-c3e89b6090ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:45:01.088193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:01.089359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:45:01.090681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:45:01.090681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:01.090681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:01.090681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 613/1228] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:01.095223 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2ff77-a115-4916-bb13-0f73ff3c98f8 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:01.194311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2ff77-a115-4916-bb13-0f73ff3c98f8 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:01.197472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:01.229860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe973b-4601-4413-911a-4b5f7f0ca9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:01.230463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe973b-4601-4413-911a-4b5f7f0ca9b6 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:01.230734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe973b-4601-4413-911a-4b5f7f0ca9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:01.230997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe973b-4601-4413-911a-4b5f7f0ca9b6 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:01.231652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe973b-4601-4413-911a-4b5f7f0ca9b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:01.231949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe973b-4601-4413-911a-4b5f7f0ca9b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:01.233275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe973b-4601-4413-911a-4b5f7f0ca9b6 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:01.244722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 616/1229] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:45:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:01.250510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be32ffc-cac5-4797-b7ce-c3e89b6090ac None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:01.260607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:01.349523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be32ffc-cac5-4797-b7ce-c3e89b6090ac None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:01.381313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a967bf-1e5d-4f4e-9a2d-9e6feb22edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:01.381924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a967bf-1e5d-4f4e-9a2d-9e6feb22edd8 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:01.382218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a967bf-1e5d-4f4e-9a2d-9e6feb22edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:01.382458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a967bf-1e5d-4f4e-9a2d-9e6feb22edd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:01.389638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a967bf-1e5d-4f4e-9a2d-9e6feb22edd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:01.391116 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a967bf-1e5d-4f4e-9a2d-9e6feb22edd8 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-e8a967bf-1e5d-4f4e-9a2d-9e6feb22edd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:45:01.403306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:01.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:01.407584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:45:01.407584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:45:01.407584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:45:01.407584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:45:01.407584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:45:01.407584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:45:01.407584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:45:01.407584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:01.407584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 614/1230] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:45:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:01.410218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe973b-4601-4413-911a-4b5f7f0ca9b6 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:01.512327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe973b-4601-4413-911a-4b5f7f0ca9b6 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:01.544703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c366a2a-896d-4448-bd5d-6c9ec5e87daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:01.545332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c366a2a-896d-4448-bd5d-6c9ec5e87daf None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:01.545612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c366a2a-896d-4448-bd5d-6c9ec5e87daf None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:01.545884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c366a2a-896d-4448-bd5d-6c9ec5e87daf None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:01.546568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c366a2a-896d-4448-bd5d-6c9ec5e87daf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:01.546872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c366a2a-896d-4448-bd5d-6c9ec5e87daf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:01.548320 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c366a2a-896d-4448-bd5d-6c9ec5e87daf None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:01.560620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 617/1231] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:45:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:01.566889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a967bf-1e5d-4f4e-9a2d-9e6feb22edd8 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:01.631902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:01.665959 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a967bf-1e5d-4f4e-9a2d-9e6feb22edd8 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:01.699473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58731943-2346-450d-a089-1f109e035e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:01.700125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58731943-2346-450d-a089-1f109e035e00 None admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:01.700404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58731943-2346-450d-a089-1f109e035e00 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:01.700676 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58731943-2346-450d-a089-1f109e035e00 None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/ad8825c294534a7c88877b48b5f8f07f/roles/4f58cf3fe09f44b1b59e224e7077ce04` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:01.714860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58731943-2346-450d-a089-1f109e035e00 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=ad8825c294534a7c88877b48b5f8f07f, role_id=4f58cf3fe09f44b1b59e224e7077ce04)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:01.716364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58731943-2346-450d-a089-1f109e035e00 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:01.797335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 615/1232] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:45:01 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/ad8825c294534a7c88877b48b5f8f07f/roles/4f58cf3fe09f44b1b59e224e7077ce04 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:45:03.712711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd269-5ce8-4e97-b389-7a08568aa6cf None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:03.713257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd269-5ce8-4e97-b389-7a08568aa6cf None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:03.713423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bd269-5ce8-4e97-b389-7a08568aa6cf None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:03.714779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 618/1233] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:45:03 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:45:03.720279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae27c35-dfc1-4ce2-afa9-1f7af7e68f81 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:03.720681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae27c35-dfc1-4ce2-afa9-1f7af7e68f81 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:03.720824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae27c35-dfc1-4ce2-afa9-1f7af7e68f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:03.739102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6ae27c35-dfc1-4ce2-afa9-1f7af7e68f81 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:45:03.740329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6ae27c35-dfc1-4ce2-afa9-1f7af7e68f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:45:03.805080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 616/1234] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:45:03 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:45:04.077737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bd269-5ce8-4e97-b389-7a08568aa6cf None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:04.103278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:04.178151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bd269-5ce8-4e97-b389-7a08568aa6cf None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:04.213915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0a868-8d6d-42d9-b4fe-c048f77bc9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:04.214534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0a868-8d6d-42d9-b4fe-c048f77bc9c3 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:04.214811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0a868-8d6d-42d9-b4fe-c048f77bc9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:04.215108 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0a868-8d6d-42d9-b4fe-c048f77bc9c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:04.224855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0a868-8d6d-42d9-b4fe-c048f77bc9c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-3bf0a868-8d6d-42d9-b4fe-c048f77bc9c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:45:04.229489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:04.231422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:04.233164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 619/1235] 10.222.0.105 () {56 vars in 1107 bytes} [Wed Aug 24 04:45:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:04.236861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae27c35-dfc1-4ce2-afa9-1f7af7e68f81 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:04.283197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:04.334725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae27c35-dfc1-4ce2-afa9-1f7af7e68f81 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:04.367263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6f1df-d870-42c6-aa12-279236e93fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:04.368832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6f1df-d870-42c6-aa12-279236e93fb4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:04.368832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6f1df-d870-42c6-aa12-279236e93fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:04.369085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6f1df-d870-42c6-aa12-279236e93fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:04.369834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6f1df-d870-42c6-aa12-279236e93fb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:04.370128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6f1df-d870-42c6-aa12-279236e93fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:04.371500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6f1df-d870-42c6-aa12-279236e93fb4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:04.379079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 617/1236] 10.222.0.105 () {56 vars in 1108 bytes} [Wed Aug 24 04:45:04 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:04.385420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0a868-8d6d-42d9-b4fe-c048f77bc9c3 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:04.469355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:04.484703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0a868-8d6d-42d9-b4fe-c048f77bc9c3 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:04.518463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc1f69-7273-4077-9f22-e18706820a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:04.519262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc1f69-7273-4077-9f22-e18706820a6b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:04.519545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc1f69-7273-4077-9f22-e18706820a6b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:04.519816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc1f69-7273-4077-9f22-e18706820a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:04.529782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc1f69-7273-4077-9f22-e18706820a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:04.531244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fc1f69-7273-4077-9f22-e18706820a6b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-19fc1f69-7273-4077-9f22-e18706820a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:45:04.545413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:04.546558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:45:04.547511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:45:04.547511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:04.547511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:04.547511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 620/1237] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:45:04.552958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6f1df-d870-42c6-aa12-279236e93fb4 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:04.654214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6f1df-d870-42c6-aa12-279236e93fb4 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:04.662763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:04.688075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46843b45-0c64-49db-b220-0e8ea3e3139c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:04.688704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46843b45-0c64-49db-b220-0e8ea3e3139c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:04.688979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46843b45-0c64-49db-b220-0e8ea3e3139c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:04.689240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46843b45-0c64-49db-b220-0e8ea3e3139c None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:04.689869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46843b45-0c64-49db-b220-0e8ea3e3139c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:04.690163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46843b45-0c64-49db-b220-0e8ea3e3139c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:04.691549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46843b45-0c64-49db-b220-0e8ea3e3139c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:04.703380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 618/1238] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:45:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:04.709441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc1f69-7273-4077-9f22-e18706820a6b None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:04.808413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fc1f69-7273-4077-9f22-e18706820a6b None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:04.841190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452b1dd-e7d5-41d0-9f33-434bb9231aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:04.841800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452b1dd-e7d5-41d0-9f33-434bb9231aff None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:04.842074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452b1dd-e7d5-41d0-9f33-434bb9231aff None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:04.842339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452b1dd-e7d5-41d0-9f33-434bb9231aff None admin] PATH_INFO: `/v3/users/placement` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:04.844944 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:04.849346 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452b1dd-e7d5-41d0-9f33-434bb9231aff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:04.850766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452b1dd-e7d5-41d0-9f33-434bb9231aff None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b452b1dd-e7d5-41d0-9f33-434bb9231aff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:45:04.862527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:04.863805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:04.864776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 621/1239] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:04.870149 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46843b45-0c64-49db-b220-0e8ea3e3139c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:04.969125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46843b45-0c64-49db-b220-0e8ea3e3139c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:05.003755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f3b25-3b3e-4af7-b0c3-7d721576dc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:05.004354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f3b25-3b3e-4af7-b0c3-7d721576dc84 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:05.004632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f3b25-3b3e-4af7-b0c3-7d721576dc84 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:05.004909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f3b25-3b3e-4af7-b0c3-7d721576dc84 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:05.005623 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f3b25-3b3e-4af7-b0c3-7d721576dc84 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:05.005927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f3b25-3b3e-4af7-b0c3-7d721576dc84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:05.007380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f3b25-3b3e-4af7-b0c3-7d721576dc84 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:05.024761 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 619/1240] 10.222.0.105 () {56 vars in 1148 bytes} [Wed Aug 24 04:45:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:05.030732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452b1dd-e7d5-41d0-9f33-434bb9231aff None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:05.130943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452b1dd-e7d5-41d0-9f33-434bb9231aff None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:05.163291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fff299-a45f-4450-9c7d-b272ac9697a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:05.163882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fff299-a45f-4450-9c7d-b272ac9697a6 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:05.164155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fff299-a45f-4450-9c7d-b272ac9697a6 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:05.164419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fff299-a45f-4450-9c7d-b272ac9697a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:05.225777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:05.232866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fff299-a45f-4450-9c7d-b272ac9697a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:05.234358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fff299-a45f-4450-9c7d-b272ac9697a6 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:05.329681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 622/1241] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:45:05 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:45:05.335777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f3b25-3b3e-4af7-b0c3-7d721576dc84 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:05.364081 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:05.433983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f3b25-3b3e-4af7-b0c3-7d721576dc84 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:05.468960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c227c12-bc8d-456e-947a-7f15a976c17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:05.469571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227c12-bc8d-456e-947a-7f15a976c17c None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:05.469843 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227c12-bc8d-456e-947a-7f15a976c17c None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:05.470106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c227c12-bc8d-456e-947a-7f15a976c17c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:05.482236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c227c12-bc8d-456e-947a-7f15a976c17c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:05.482236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c227c12-bc8d-456e-947a-7f15a976c17c None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-1c227c12-bc8d-456e-947a-7f15a976c17c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:45:05.495548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:05.496709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:45:05.497698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:45:05.497698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:05.497698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:05.497698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 620/1242] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:05.502652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fff299-a45f-4450-9c7d-b272ac9697a6 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:05.593977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:05.599498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fff299-a45f-4450-9c7d-b272ac9697a6 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:05.631597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c980aa-059b-4798-97d0-7291b8088b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:05.632193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c980aa-059b-4798-97d0-7291b8088b98 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:05.632466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c980aa-059b-4798-97d0-7291b8088b98 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:05.632732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c980aa-059b-4798-97d0-7291b8088b98 None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:05.633362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c980aa-059b-4798-97d0-7291b8088b98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:05.633657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c980aa-059b-4798-97d0-7291b8088b98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:05.635794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c980aa-059b-4798-97d0-7291b8088b98 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:05.646657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 623/1243] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:45:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:05.653311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c227c12-bc8d-456e-947a-7f15a976c17c None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:05.730685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:05.751477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c227c12-bc8d-456e-947a-7f15a976c17c None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:05.784855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50b2f7-41e9-45d1-bdd1-2bb20f91ed37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:05.785582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50b2f7-41e9-45d1-bdd1-2bb20f91ed37 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:05.785863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50b2f7-41e9-45d1-bdd1-2bb20f91ed37 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:05.786134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50b2f7-41e9-45d1-bdd1-2bb20f91ed37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:05.793292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50b2f7-41e9-45d1-bdd1-2bb20f91ed37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:05.794832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50b2f7-41e9-45d1-bdd1-2bb20f91ed37 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-6a50b2f7-41e9-45d1-bdd1-2bb20f91ed37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:45:05.807653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:05.808739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:05.809894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:45:05.809894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:45:05.809894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:45:05.809894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:45:05.809894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:45:05.809894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:45:05.809894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:45:05.809894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:05.809894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 621/1244] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:05.815012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c980aa-059b-4798-97d0-7291b8088b98 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:05.917788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c980aa-059b-4798-97d0-7291b8088b98 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:05.950231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0bd75-ae9f-46e3-b534-e3809e561a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:05.950844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0bd75-ae9f-46e3-b534-e3809e561a34 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:05.951162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0bd75-ae9f-46e3-b534-e3809e561a34 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:05.951437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0bd75-ae9f-46e3-b534-e3809e561a34 None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:05.957118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0bd75-ae9f-46e3-b534-e3809e561a34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:05.957118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0bd75-ae9f-46e3-b534-e3809e561a34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:05.957118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0bd75-ae9f-46e3-b534-e3809e561a34 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:05.967178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 624/1245] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:05.973395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50b2f7-41e9-45d1-bdd1-2bb20f91ed37 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:06.073252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50b2f7-41e9-45d1-bdd1-2bb20f91ed37 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:06.106216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8e0c6-4f5a-4dd1-9b72-84341de5ce00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:06.106810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8e0c6-4f5a-4dd1-9b72-84341de5ce00 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:06.107118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8e0c6-4f5a-4dd1-9b72-84341de5ce00 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:06.107394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8e0c6-4f5a-4dd1-9b72-84341de5ce00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:06.108113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8e0c6-4f5a-4dd1-9b72-84341de5ce00 None admin] RBAC: Adding query filter params (role.id=4f58cf3fe09f44b1b59e224e7077ce04, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=ad8825c294534a7c88877b48b5f8f07f) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:06.108419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8e0c6-4f5a-4dd1-9b72-84341de5ce00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:06.109828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8e0c6-4f5a-4dd1-9b72-84341de5ce00 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:06.115541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 622/1246] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:45:05 2022] GET /identity/v3/role_assignments?user.id=ad8825c294534a7c88877b48b5f8f07f&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=4f58cf3fe09f44b1b59e224e7077ce04 => generated 610 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:08.083339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49baf8b6-5fb7-4ba3-b53d-493dda53cc15 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:08.083887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49baf8b6-5fb7-4ba3-b53d-493dda53cc15 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:08.083887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49baf8b6-5fb7-4ba3-b53d-493dda53cc15 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:08.085190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 625/1247] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:45:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:45:08.090606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b47a4-949d-404d-98ed-5baa338c7204 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:08.091053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b47a4-949d-404d-98ed-5baa338c7204 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:08.091239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b47a4-949d-404d-98ed-5baa338c7204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:08.097510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:08.109371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-766b47a4-949d-404d-98ed-5baa338c7204 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:45:08.110559 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-766b47a4-949d-404d-98ed-5baa338c7204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:45:08.176940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 623/1248] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:45:08 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:45:08.446930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49baf8b6-5fb7-4ba3-b53d-493dda53cc15 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:08.465147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:08.547517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49baf8b6-5fb7-4ba3-b53d-493dda53cc15 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:08.580866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c034e-d6b6-4f8b-866b-0af755b54014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:08.581485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c034e-d6b6-4f8b-866b-0af755b54014 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:08.581758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c034e-d6b6-4f8b-866b-0af755b54014 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:08.582026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49c034e-d6b6-4f8b-866b-0af755b54014 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:08.591854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49c034e-d6b6-4f8b-866b-0af755b54014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-e49c034e-d6b6-4f8b-866b-0af755b54014 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:45:08.596438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:08.597584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:08.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 626/1249] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:45:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:08.604364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b47a4-949d-404d-98ed-5baa338c7204 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:08.702452 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b47a4-949d-404d-98ed-5baa338c7204 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:08.733053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:08.737789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df6d25-3c7c-4c2f-bc41-f7b7998c9277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:08.738441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df6d25-3c7c-4c2f-bc41-f7b7998c9277 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:08.738687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df6d25-3c7c-4c2f-bc41-f7b7998c9277 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:08.738949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df6d25-3c7c-4c2f-bc41-f7b7998c9277 None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:08.739644 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df6d25-3c7c-4c2f-bc41-f7b7998c9277 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:08.739947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df6d25-3c7c-4c2f-bc41-f7b7998c9277 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:08.741299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df6d25-3c7c-4c2f-bc41-f7b7998c9277 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:08.750215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 624/1250] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:45:08 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:08.756047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c034e-d6b6-4f8b-866b-0af755b54014 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:08.836840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:08.860523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49c034e-d6b6-4f8b-866b-0af755b54014 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:08.893084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5a685-ba17-451d-8cb1-53561f85cc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:08.893864 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5a685-ba17-451d-8cb1-53561f85cc06 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:08.894222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5a685-ba17-451d-8cb1-53561f85cc06 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:08.894574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5a685-ba17-451d-8cb1-53561f85cc06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:08.906241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5a685-ba17-451d-8cb1-53561f85cc06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:08.907707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5a685-ba17-451d-8cb1-53561f85cc06 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-f1a5a685-ba17-451d-8cb1-53561f85cc06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:45:08.922546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:08.923767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:45:08.924906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:45:08.924906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:08.924906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:08.924906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 627/1251] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:08.929567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df6d25-3c7c-4c2f-bc41-f7b7998c9277 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:09.029479 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df6d25-3c7c-4c2f-bc41-f7b7998c9277 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:09.062592 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb22fec-7247-4e0d-827f-dfd60bdba94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:09.063219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb22fec-7247-4e0d-827f-dfd60bdba94a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:09.063499 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb22fec-7247-4e0d-827f-dfd60bdba94a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:09.063767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb22fec-7247-4e0d-827f-dfd60bdba94a None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:09.064396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb22fec-7247-4e0d-827f-dfd60bdba94a None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:09.064694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb22fec-7247-4e0d-827f-dfd60bdba94a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:09.066085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb22fec-7247-4e0d-827f-dfd60bdba94a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:09.077552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 625/1252] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:45:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:09.083991 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5a685-ba17-451d-8cb1-53561f85cc06 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:09.189789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5a685-ba17-451d-8cb1-53561f85cc06 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:09.222147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:09.229903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce030d-e7a6-4e5a-9513-29fca8e8d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:09.230535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce030d-e7a6-4e5a-9513-29fca8e8d63f None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:09.230811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce030d-e7a6-4e5a-9513-29fca8e8d63f None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:09.231114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce030d-e7a6-4e5a-9513-29fca8e8d63f None admin] PATH_INFO: `/v3/users/placement` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:09.238664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce030d-e7a6-4e5a-9513-29fca8e8d63f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:09.240237 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce030d-e7a6-4e5a-9513-29fca8e8d63f None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-78ce030d-e7a6-4e5a-9513-29fca8e8d63f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:45:09.252103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:09.253255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:09.254432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 628/1253] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:09.259183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb22fec-7247-4e0d-827f-dfd60bdba94a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:09.314341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:09.358347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb22fec-7247-4e0d-827f-dfd60bdba94a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:09.392124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c7856-b7a0-4df4-bcc2-82046bc8a028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:09.392722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c7856-b7a0-4df4-bcc2-82046bc8a028 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:09.393000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c7856-b7a0-4df4-bcc2-82046bc8a028 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:09.393269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c7856-b7a0-4df4-bcc2-82046bc8a028 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:09.393977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c7856-b7a0-4df4-bcc2-82046bc8a028 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:09.394278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c7856-b7a0-4df4-bcc2-82046bc8a028 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:09.395721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c7856-b7a0-4df4-bcc2-82046bc8a028 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:09.412681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 626/1254] 10.222.0.105 () {56 vars in 1148 bytes} [Wed Aug 24 04:45:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:09.418600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce030d-e7a6-4e5a-9513-29fca8e8d63f None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:09.520158 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce030d-e7a6-4e5a-9513-29fca8e8d63f None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:09.554116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cef46-d5e5-41ad-9a57-73c3843d9656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:09.554699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cef46-d5e5-41ad-9a57-73c3843d9656 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:09.554981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cef46-d5e5-41ad-9a57-73c3843d9656 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:09.555328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cef46-d5e5-41ad-9a57-73c3843d9656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:09.611476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:09.623903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cef46-d5e5-41ad-9a57-73c3843d9656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:09.625326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cef46-d5e5-41ad-9a57-73c3843d9656 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:09.722028 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 629/1255] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:45:09 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:45:09.727896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c7856-b7a0-4df4-bcc2-82046bc8a028 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:09.825541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c7856-b7a0-4df4-bcc2-82046bc8a028 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:09.838708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:09.859154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16323a6b-e485-4e3d-a00b-95c4a4c71678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:09.859801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323a6b-e485-4e3d-a00b-95c4a4c71678 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:09.860078 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323a6b-e485-4e3d-a00b-95c4a4c71678 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:09.860344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16323a6b-e485-4e3d-a00b-95c4a4c71678 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:09.872824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16323a6b-e485-4e3d-a00b-95c4a4c71678 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:09.872824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16323a6b-e485-4e3d-a00b-95c4a4c71678 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-16323a6b-e485-4e3d-a00b-95c4a4c71678 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:45:09.887464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:09.888615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:45:09.890022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:45:09.890022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:09.890022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:09.890022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 627/1256] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:09.894326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cef46-d5e5-41ad-9a57-73c3843d9656 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:09.979327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:09.990792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cef46-d5e5-41ad-9a57-73c3843d9656 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:10.024417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7984d3-f416-42c8-803c-49f32a45dd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:10.025008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7984d3-f416-42c8-803c-49f32a45dd8c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:10.025281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7984d3-f416-42c8-803c-49f32a45dd8c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:10.025547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7984d3-f416-42c8-803c-49f32a45dd8c None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:10.026458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7984d3-f416-42c8-803c-49f32a45dd8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:10.026754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7984d3-f416-42c8-803c-49f32a45dd8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:10.028134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7984d3-f416-42c8-803c-49f32a45dd8c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:10.039153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 630/1257] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:45:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:10.045421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16323a6b-e485-4e3d-a00b-95c4a4c71678 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:10.144803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16323a6b-e485-4e3d-a00b-95c4a4c71678 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:10.178126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fd4be-9278-4e6a-b7fa-1db1af81d1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:10.178749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fd4be-9278-4e6a-b7fa-1db1af81d1f7 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:10.179047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fd4be-9278-4e6a-b7fa-1db1af81d1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:10.179325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940fd4be-9278-4e6a-b7fa-1db1af81d1f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:10.188353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fd4be-9278-4e6a-b7fa-1db1af81d1f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:10.188353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940fd4be-9278-4e6a-b7fa-1db1af81d1f7 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-940fd4be-9278-4e6a-b7fa-1db1af81d1f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:45:10.201069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:10.203830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:10.204934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:45:10.204934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:45:10.204934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:45:10.204934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:45:10.204934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:45:10.204934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:45:10.204934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:45:10.204934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:10.204934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 628/1258] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:45:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:10.208309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7984d3-f416-42c8-803c-49f32a45dd8c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:10.305455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7984d3-f416-42c8-803c-49f32a45dd8c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:10.339533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6ea1a5-0117-4c8d-91bb-cc3ba4fffd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:10.340133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ea1a5-0117-4c8d-91bb-cc3ba4fffd1a None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:10.340432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ea1a5-0117-4c8d-91bb-cc3ba4fffd1a None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:10.340683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6ea1a5-0117-4c8d-91bb-cc3ba4fffd1a None admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:10.341378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ea1a5-0117-4c8d-91bb-cc3ba4fffd1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:10.341692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ea1a5-0117-4c8d-91bb-cc3ba4fffd1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:10.343167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6ea1a5-0117-4c8d-91bb-cc3ba4fffd1a None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:10.355135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 631/1259] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:10.361417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fd4be-9278-4e6a-b7fa-1db1af81d1f7 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:10.387797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:10.460769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940fd4be-9278-4e6a-b7fa-1db1af81d1f7 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:10.494024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999264cd-7a51-4f0d-aaac-dce54d066eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:10.494642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999264cd-7a51-4f0d-aaac-dce54d066eb3 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:10.494910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999264cd-7a51-4f0d-aaac-dce54d066eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:10.495208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999264cd-7a51-4f0d-aaac-dce54d066eb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:10.495917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999264cd-7a51-4f0d-aaac-dce54d066eb3 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=ad8825c294534a7c88877b48b5f8f07f) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:10.496220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999264cd-7a51-4f0d-aaac-dce54d066eb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:10.497636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999264cd-7a51-4f0d-aaac-dce54d066eb3 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:10.502775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 629/1260] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:45:10 2022] GET /identity/v3/role_assignments?user.id=ad8825c294534a7c88877b48b5f8f07f&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 260 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:12.400545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd650-5046-4d6d-ad67-39a8b44988bf None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:12.401161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd650-5046-4d6d-ad67-39a8b44988bf None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:12.401257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1fd650-5046-4d6d-ad67-39a8b44988bf None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:12.402647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 632/1261] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:45:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:45:12.407871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9fa054-495a-4d27-99fb-43609ec9494f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:12.408313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9fa054-495a-4d27-99fb-43609ec9494f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:12.408430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9fa054-495a-4d27-99fb-43609ec9494f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:12.427302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-bc9fa054-495a-4d27-99fb-43609ec9494f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:45:12.428512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-bc9fa054-495a-4d27-99fb-43609ec9494f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:45:12.494293 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 630/1262] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:45:12 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:45:12.763577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd650-5046-4d6d-ad67-39a8b44988bf None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:12.774251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:12.863853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1fd650-5046-4d6d-ad67-39a8b44988bf None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:12.896902 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd5884-0779-4b97-a432-5dbe05979aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:12.897524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd5884-0779-4b97-a432-5dbe05979aaa None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:12.897801 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd5884-0779-4b97-a432-5dbe05979aaa None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:12.898068 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd5884-0779-4b97-a432-5dbe05979aaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:12.909506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd5884-0779-4b97-a432-5dbe05979aaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-dccd5884-0779-4b97-a432-5dbe05979aaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:45:12.914140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:12.915318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:12.916181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 633/1263] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:45:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:12.922355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9fa054-495a-4d27-99fb-43609ec9494f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:12.940411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:13.017685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9fa054-495a-4d27-99fb-43609ec9494f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:13.050501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bd7f9-ae87-466c-b8c9-84e3d64963ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:13.051133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bd7f9-ae87-466c-b8c9-84e3d64963ff None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:13.051413 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bd7f9-ae87-466c-b8c9-84e3d64963ff None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:13.051702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195bd7f9-ae87-466c-b8c9-84e3d64963ff None admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:13.052392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195bd7f9-ae87-466c-b8c9-84e3d64963ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:13.052693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195bd7f9-ae87-466c-b8c9-84e3d64963ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:13.054051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195bd7f9-ae87-466c-b8c9-84e3d64963ff None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:13.061947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 631/1264] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:45:12 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:13.067827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd5884-0779-4b97-a432-5dbe05979aaa None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:13.134793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:13.168677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd5884-0779-4b97-a432-5dbe05979aaa None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:13.201899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca185e03-a5e1-4126-be41-21f0c0a662aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:13.202537 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185e03-a5e1-4126-be41-21f0c0a662aa None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:13.202815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185e03-a5e1-4126-be41-21f0c0a662aa None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:13.203120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185e03-a5e1-4126-be41-21f0c0a662aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:13.214359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca185e03-a5e1-4126-be41-21f0c0a662aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:13.215853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca185e03-a5e1-4126-be41-21f0c0a662aa None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-ca185e03-a5e1-4126-be41-21f0c0a662aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:45:13.229788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:13.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:45:13.241291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:45:13.241291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:13.241291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:13.241291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 634/1265] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:13.241570 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bd7f9-ae87-466c-b8c9-84e3d64963ff None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:13.317850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:13.337290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195bd7f9-ae87-466c-b8c9-84e3d64963ff None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:13.370959 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f89263-5f78-4935-83bb-a170798565c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:13.371645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f89263-5f78-4935-83bb-a170798565c4 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:13.371927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f89263-5f78-4935-83bb-a170798565c4 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:13.372200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f89263-5f78-4935-83bb-a170798565c4 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:13.372836 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f89263-5f78-4935-83bb-a170798565c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:13.373136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f89263-5f78-4935-83bb-a170798565c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:13.374489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f89263-5f78-4935-83bb-a170798565c4 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:13.386506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 632/1266] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:45:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:13.392252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca185e03-a5e1-4126-be41-21f0c0a662aa None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:13.489070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca185e03-a5e1-4126-be41-21f0c0a662aa None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:13.509256 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:13.523964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a39f3-cf0b-466d-93d9-53fb372361f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:13.524588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a39f3-cf0b-466d-93d9-53fb372361f8 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:13.524865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a39f3-cf0b-466d-93d9-53fb372361f8 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:13.525136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a39f3-cf0b-466d-93d9-53fb372361f8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:13.532468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a39f3-cf0b-466d-93d9-53fb372361f8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:13.533894 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a39f3-cf0b-466d-93d9-53fb372361f8 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-805a39f3-cf0b-466d-93d9-53fb372361f8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:45:13.544615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:13.545730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:13.546752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 635/1267] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:13.551833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f89263-5f78-4935-83bb-a170798565c4 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:13.649049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f89263-5f78-4935-83bb-a170798565c4 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:13.681472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4f942-53ee-432b-bf30-9a279234ef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:13.682059 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4f942-53ee-432b-bf30-9a279234ef10 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:13.682970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4f942-53ee-432b-bf30-9a279234ef10 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:13.682970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4f942-53ee-432b-bf30-9a279234ef10 None admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:13.683993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4f942-53ee-432b-bf30-9a279234ef10 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:13.684339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4f942-53ee-432b-bf30-9a279234ef10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:13.685743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4f942-53ee-432b-bf30-9a279234ef10 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:13.698052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:13.706536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 633/1268] 10.222.0.105 () {56 vars in 1148 bytes} [Wed Aug 24 04:45:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:13.712132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a39f3-cf0b-466d-93d9-53fb372361f8 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:13.813448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a39f3-cf0b-466d-93d9-53fb372361f8 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:13.846980 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfa10a-4c8a-4a25-a4ab-7cb642bbc810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:13.851000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfa10a-4c8a-4a25-a4ab-7cb642bbc810 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:13.851000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfa10a-4c8a-4a25-a4ab-7cb642bbc810 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:13.851000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfa10a-4c8a-4a25-a4ab-7cb642bbc810 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:13.859333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfa10a-4c8a-4a25-a4ab-7cb642bbc810 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:13.860768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfa10a-4c8a-4a25-a4ab-7cb642bbc810 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-eebfa10a-4c8a-4a25-a4ab-7cb642bbc810 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:45:13.875092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:13.883058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:45:13.884201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:45:13.884201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:13.884201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:13.884201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 636/1269] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:13.884480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4f942-53ee-432b-bf30-9a279234ef10 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:13.980678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4f942-53ee-432b-bf30-9a279234ef10 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:14.014450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc974b-72ee-4a70-96d3-17d26ff758d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:14.015127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc974b-72ee-4a70-96d3-17d26ff758d0 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:14.015406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc974b-72ee-4a70-96d3-17d26ff758d0 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:14.015674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cc974b-72ee-4a70-96d3-17d26ff758d0 None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:14.016303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc974b-72ee-4a70-96d3-17d26ff758d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:14.016597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc974b-72ee-4a70-96d3-17d26ff758d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:14.017944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cc974b-72ee-4a70-96d3-17d26ff758d0 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:14.029134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 634/1270] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:45:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:14.035217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfa10a-4c8a-4a25-a4ab-7cb642bbc810 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:14.067685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:14.138117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfa10a-4c8a-4a25-a4ab-7cb642bbc810 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:14.171403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb598-7806-45b6-9818-965c19e3aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:14.172029 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb598-7806-45b6-9818-965c19e3aac7 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:14.172351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb598-7806-45b6-9818-965c19e3aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:14.172642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecdb598-7806-45b6-9818-965c19e3aac7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:14.179430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecdb598-7806-45b6-9818-965c19e3aac7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:14.180850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecdb598-7806-45b6-9818-965c19e3aac7 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-1ecdb598-7806-45b6-9818-965c19e3aac7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:45:14.195329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:14.196386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:14.197503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:45:14.197503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:45:14.197503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:45:14.197503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:45:14.197503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:45:14.197503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:45:14.197503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:45:14.197503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:14.197503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 637/1271] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:45:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:14.200232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc974b-72ee-4a70-96d3-17d26ff758d0 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:14.257857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:14.297779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cc974b-72ee-4a70-96d3-17d26ff758d0 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:14.330556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c2b0b-9534-42f8-a115-3cc23db0151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:14.331443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2b0b-9534-42f8-a115-3cc23db0151d None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:14.331721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2b0b-9534-42f8-a115-3cc23db0151d None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:14.331992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2b0b-9534-42f8-a115-3cc23db0151d None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:14.332670 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c2b0b-9534-42f8-a115-3cc23db0151d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:14.332971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c2b0b-9534-42f8-a115-3cc23db0151d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:14.334398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c2b0b-9534-42f8-a115-3cc23db0151d None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:14.346101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 635/1272] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:45:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:14.352179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb598-7806-45b6-9818-965c19e3aac7 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:14.444852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:14.453564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecdb598-7806-45b6-9818-965c19e3aac7 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:14.487316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a8c60-6783-4ccb-b7b4-807afed7b658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:14.487965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a8c60-6783-4ccb-b7b4-807afed7b658 None admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:14.488242 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a8c60-6783-4ccb-b7b4-807afed7b658 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:14.488514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a8c60-6783-4ccb-b7b4-807afed7b658 None admin] PATH_INFO: `/v3/projects/d6a64369509349b4b65a772fb6057599/users/ad8825c294534a7c88877b48b5f8f07f/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:14.490454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a8c60-6783-4ccb-b7b4-807afed7b658 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a64369509349b4b65a772fb6057599, user_id=ad8825c294534a7c88877b48b5f8f07f, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:14.491903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a8c60-6783-4ccb-b7b4-807afed7b658 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:14.566036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 638/1273] 10.222.0.105 () {58 vars in 1444 bytes} [Wed Aug 24 04:45:14 2022] PUT /identity/v3/projects/d6a64369509349b4b65a772fb6057599/users/ad8825c294534a7c88877b48b5f8f07f/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:45:16.450659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63697354-1884-47f9-9dca-dc60224e003c None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:16.451237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63697354-1884-47f9-9dca-dc60224e003c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:16.451237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63697354-1884-47f9-9dca-dc60224e003c None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:16.452624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 636/1274] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:45:16 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:45:16.458146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb1b2f-3563-41d9-ba14-b6ca1152484e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:16.458552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb1b2f-3563-41d9-ba14-b6ca1152484e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:16.458711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb1b2f-3563-41d9-ba14-b6ca1152484e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:16.476397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-e7cb1b2f-3563-41d9-ba14-b6ca1152484e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:45:16.477582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e7cb1b2f-3563-41d9-ba14-b6ca1152484e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:45:16.543271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 639/1275] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:45:16 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:45:16.815772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63697354-1884-47f9-9dca-dc60224e003c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:16.915598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63697354-1884-47f9-9dca-dc60224e003c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:16.933720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:16.949628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397c595-61a7-449f-887f-889041a8d6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:16.950247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397c595-61a7-449f-887f-889041a8d6f9 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:16.950525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397c595-61a7-449f-887f-889041a8d6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:16.950795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397c595-61a7-449f-887f-889041a8d6f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:16.960735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397c595-61a7-449f-887f-889041a8d6f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-c397c595-61a7-449f-887f-889041a8d6f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 04:45:16.965406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:16.966537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:16.967522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 637/1276] 10.222.0.105 () {56 vars in 1101 bytes} [Wed Aug 24 04:45:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:45:16.972637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb1b2f-3563-41d9-ba14-b6ca1152484e None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:17.053817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:17.071611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb1b2f-3563-41d9-ba14-b6ca1152484e None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:17.106089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec300d8e-b642-4a67-a382-74354252eb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:17.106696 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec300d8e-b642-4a67-a382-74354252eb58 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:17.106985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec300d8e-b642-4a67-a382-74354252eb58 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:17.107281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec300d8e-b642-4a67-a382-74354252eb58 None admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:17.107944 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec300d8e-b642-4a67-a382-74354252eb58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:17.108245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec300d8e-b642-4a67-a382-74354252eb58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:17.109588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec300d8e-b642-4a67-a382-74354252eb58 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:17.117425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 640/1277] 10.222.0.105 () {56 vars in 1104 bytes} [Wed Aug 24 04:45:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:17.123746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397c595-61a7-449f-887f-889041a8d6f9 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:17.226762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397c595-61a7-449f-887f-889041a8d6f9 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:17.260002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0389ca-8299-4f1b-9ee6-689efcddbc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:17.260617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0389ca-8299-4f1b-9ee6-689efcddbc7f None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:17.260895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0389ca-8299-4f1b-9ee6-689efcddbc7f None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:17.261275 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0389ca-8299-4f1b-9ee6-689efcddbc7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:17.271264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0389ca-8299-4f1b-9ee6-689efcddbc7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:17.279311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0389ca-8299-4f1b-9ee6-689efcddbc7f None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-3a0389ca-8299-4f1b-9ee6-689efcddbc7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:45:17.287406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:17.288508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:45:17.289646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:45:17.289646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:17.289646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:17.289646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 638/1278] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:17.295293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec300d8e-b642-4a67-a382-74354252eb58 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:17.395082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec300d8e-b642-4a67-a382-74354252eb58 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:17.427575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d15d3-0162-49b6-97ab-d13f6fa56eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:17.428197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d15d3-0162-49b6-97ab-d13f6fa56eff None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:17.428474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d15d3-0162-49b6-97ab-d13f6fa56eff None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:17.428742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d15d3-0162-49b6-97ab-d13f6fa56eff None admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:17.429371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d15d3-0162-49b6-97ab-d13f6fa56eff None admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:17.429671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d15d3-0162-49b6-97ab-d13f6fa56eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:17.431048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147d15d3-0162-49b6-97ab-d13f6fa56eff None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:17.443303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 641/1279] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:45:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:17.449801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0389ca-8299-4f1b-9ee6-689efcddbc7f None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:17.481578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:17.548662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0389ca-8299-4f1b-9ee6-689efcddbc7f None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:17.581887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffe22c-7a10-4351-a8e6-11768f67d89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:17.582512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffe22c-7a10-4351-a8e6-11768f67d89b None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:17.582789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffe22c-7a10-4351-a8e6-11768f67d89b None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:17.583083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ffe22c-7a10-4351-a8e6-11768f67d89b None admin] PATH_INFO: `/v3/users/placement` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:17.590292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffe22c-7a10-4351-a8e6-11768f67d89b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:17.591734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ffe22c-7a10-4351-a8e6-11768f67d89b None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-79ffe22c-7a10-4351-a8e6-11768f67d89b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 04:45:17.602750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:17.616239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:17.617348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 639/1280] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:17.617965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d15d3-0162-49b6-97ab-d13f6fa56eff None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:17.617965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:17.709506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d15d3-0162-49b6-97ab-d13f6fa56eff None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:17.742160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cbc21-1c47-4189-9528-99c14a9b569c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:17.742759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cbc21-1c47-4189-9528-99c14a9b569c None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:17.743059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cbc21-1c47-4189-9528-99c14a9b569c None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:17.743335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54cbc21-1c47-4189-9528-99c14a9b569c None admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:17.744050 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cbc21-1c47-4189-9528-99c14a9b569c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:17.744351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cbc21-1c47-4189-9528-99c14a9b569c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:17.746416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54cbc21-1c47-4189-9528-99c14a9b569c None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:17.763999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 642/1281] 10.222.0.105 () {56 vars in 1148 bytes} [Wed Aug 24 04:45:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:17.770053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffe22c-7a10-4351-a8e6-11768f67d89b None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:17.869256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:17.878633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ffe22c-7a10-4351-a8e6-11768f67d89b None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:17.911140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e98282-6eff-4524-ab48-cdd1802fe64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:17.911734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e98282-6eff-4524-ab48-cdd1802fe64a None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:17.912009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e98282-6eff-4524-ab48-cdd1802fe64a None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:17.912275 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e98282-6eff-4524-ab48-cdd1802fe64a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:17.977301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e98282-6eff-4524-ab48-cdd1802fe64a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:17.978682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e98282-6eff-4524-ab48-cdd1802fe64a None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:18.065219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:18.075468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 640/1282] 10.222.0.105 () {58 vars in 1287 bytes} [Wed Aug 24 04:45:17 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 04:45:18.081312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cbc21-1c47-4189-9528-99c14a9b569c None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:18.141997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:18.180762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54cbc21-1c47-4189-9528-99c14a9b569c None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:18.213790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6fa2b-1913-4d5d-b2cf-881323a58413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:18.215099 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6fa2b-1913-4d5d-b2cf-881323a58413 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:18.215383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6fa2b-1913-4d5d-b2cf-881323a58413 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:18.215649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6fa2b-1913-4d5d-b2cf-881323a58413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:18.225639 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6fa2b-1913-4d5d-b2cf-881323a58413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:18.227092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6fa2b-1913-4d5d-b2cf-881323a58413 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-b9f6fa2b-1913-4d5d-b2cf-881323a58413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 04:45:18.241985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:18.243235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 04:45:18.244461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 04:45:18.244461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 04:45:18.244461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:18.244461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 643/1283] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 04:45:18.248890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e98282-6eff-4524-ab48-cdd1802fe64a None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:18.355719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e98282-6eff-4524-ab48-cdd1802fe64a None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:18.401518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2174045-1e38-4298-98b8-2a0225e5f3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:18.402203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174045-1e38-4298-98b8-2a0225e5f3fc None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:18.402658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174045-1e38-4298-98b8-2a0225e5f3fc None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:18.403075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2174045-1e38-4298-98b8-2a0225e5f3fc None admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:18.403740 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2174045-1e38-4298-98b8-2a0225e5f3fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:18.404042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2174045-1e38-4298-98b8-2a0225e5f3fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:18.405463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2174045-1e38-4298-98b8-2a0225e5f3fc None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:18.423876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 641/1284] 10.222.0.105 () {56 vars in 1114 bytes} [Wed Aug 24 04:45:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:18.425157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6fa2b-1913-4d5d-b2cf-881323a58413 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:18.535839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6fa2b-1913-4d5d-b2cf-881323a58413 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:18.551453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:18.573018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9674adc-34c0-41a7-934c-fa180274eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:18.573771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9674adc-34c0-41a7-934c-fa180274eca7 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:18.574075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9674adc-34c0-41a7-934c-fa180274eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:18.574860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9674adc-34c0-41a7-934c-fa180274eca7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:18.583380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9674adc-34c0-41a7-934c-fa180274eca7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:18.584961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9674adc-34c0-41a7-934c-fa180274eca7 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application [None req-f9674adc-34c0-41a7-934c-fa180274eca7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 04:45:18.597077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:18.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:18.599239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 04:45:18.599239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 04:45:18.599239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 04:45:18.599239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 04:45:18.599239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 04:45:18.599239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 04:45:18.599239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 04:45:18.599239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: ERROR keystone.server.flask.application Aug 24 04:45:18.599239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 644/1285] 10.222.0.105 () {56 vars in 1116 bytes} [Wed Aug 24 04:45:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:18.604433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2174045-1e38-4298-98b8-2a0225e5f3fc None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:18.657837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:18.713681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2174045-1e38-4298-98b8-2a0225e5f3fc None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:18.748649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0bf04-85d4-49aa-b2e4-f6da16664c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:18.749272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0bf04-85d4-49aa-b2e4-f6da16664c28 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:18.749554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0bf04-85d4-49aa-b2e4-f6da16664c28 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:18.749824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0bf04-85d4-49aa-b2e4-f6da16664c28 None admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:18.750506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0bf04-85d4-49aa-b2e4-f6da16664c28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:18.750811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0bf04-85d4-49aa-b2e4-f6da16664c28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:18.752286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0bf04-85d4-49aa-b2e4-f6da16664c28 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:18.764234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 642/1286] 10.222.0.105 () {56 vars in 1153 bytes} [Wed Aug 24 04:45:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:18.770365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9674adc-34c0-41a7-934c-fa180274eca7 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:18.869343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9674adc-34c0-41a7-934c-fa180274eca7 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:18.901516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c343f2-0cf4-4542-bdfc-447851ec9252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:18.902671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c343f2-0cf4-4542-bdfc-447851ec9252 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:18.903072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c343f2-0cf4-4542-bdfc-447851ec9252 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:18.903350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c343f2-0cf4-4542-bdfc-447851ec9252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:18.904071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c343f2-0cf4-4542-bdfc-447851ec9252 None admin] RBAC: Adding query filter params (role.id=72df4feaecfe48d1a12346e9c9ede92a, scope.project.id=d6a64369509349b4b65a772fb6057599, user.id=ad8825c294534a7c88877b48b5f8f07f) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:18.904377 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c343f2-0cf4-4542-bdfc-447851ec9252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:18.905783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c343f2-0cf4-4542-bdfc-447851ec9252 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:18.911389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 645/1287] 10.222.0.105 () {56 vars in 1379 bytes} [Wed Aug 24 04:45:18 2022] GET /identity/v3/role_assignments?user.id=ad8825c294534a7c88877b48b5f8f07f&scope.project.id=d6a64369509349b4b65a772fb6057599&role.id=72df4feaecfe48d1a12346e9c9ede92a => generated 610 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:20.869032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f31d6-a7c9-49c8-8f6e-eb826d2bf363 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:20.869581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f31d6-a7c9-49c8-8f6e-eb826d2bf363 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:20.869581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815f31d6-a7c9-49c8-8f6e-eb826d2bf363 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:20.870914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 643/1288] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:45:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:45:20.876958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f60ba-07c1-456c-8885-a26e985ae7ed None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:20.877358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f60ba-07c1-456c-8885-a26e985ae7ed None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:20.877511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f60ba-07c1-456c-8885-a26e985ae7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:20.895741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-e45f60ba-07c1-456c-8885-a26e985ae7ed None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:45:20.896919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e45f60ba-07c1-456c-8885-a26e985ae7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:45:20.925157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:20.962325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 646/1289] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:45:20 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:45:21.233640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f31d6-a7c9-49c8-8f6e-eb826d2bf363 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:21.334112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815f31d6-a7c9-49c8-8f6e-eb826d2bf363 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:21.345357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:21.368791 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbc82c-109c-455f-8a0c-7b7fb6de3998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:21.369420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbc82c-109c-455f-8a0c-7b7fb6de3998 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:21.369694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbc82c-109c-455f-8a0c-7b7fb6de3998 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:21.369962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbc82c-109c-455f-8a0c-7b7fb6de3998 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:21.370533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbc82c-109c-455f-8a0c-7b7fb6de3998 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:21.371949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbc82c-109c-455f-8a0c-7b7fb6de3998 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-cabbc82c-109c-455f-8a0c-7b7fb6de3998 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:21.380544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:21.381810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:21.382854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 644/1290] 10.222.0.105 () {56 vars in 1122 bytes} [Wed Aug 24 04:45:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:21.388731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f60ba-07c1-456c-8885-a26e985ae7ed None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:21.486062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f60ba-07c1-456c-8885-a26e985ae7ed None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:21.520468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d4cfc-61f5-4cd2-b82e-5f9cb9e4771b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:21.521067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d4cfc-61f5-4cd2-b82e-5f9cb9e4771b None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:21.521343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d4cfc-61f5-4cd2-b82e-5f9cb9e4771b None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:21.521612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d4cfc-61f5-4cd2-b82e-5f9cb9e4771b None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:21.522240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d4cfc-61f5-4cd2-b82e-5f9cb9e4771b None admin] RBAC: Adding query filter params (name=placement) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:21.522540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d4cfc-61f5-4cd2-b82e-5f9cb9e4771b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:21.523920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d4cfc-61f5-4cd2-b82e-5f9cb9e4771b None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:21.529259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 647/1291] 10.222.0.105 () {56 vars in 1121 bytes} [Wed Aug 24 04:45:21 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:21.535350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbc82c-109c-455f-8a0c-7b7fb6de3998 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:21.632445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbc82c-109c-455f-8a0c-7b7fb6de3998 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:21.665362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c2e694-96c4-4b26-92c9-34641ae8c2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:21.665955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2e694-96c4-4b26-92c9-34641ae8c2d6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:21.666232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2e694-96c4-4b26-92c9-34641ae8c2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:21.666503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c2e694-96c4-4b26-92c9-34641ae8c2d6 None admin] PATH_INFO: `/v3/services` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:21.667164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2e694-96c4-4b26-92c9-34641ae8c2d6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:21.667506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2e694-96c4-4b26-92c9-34641ae8c2d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:21.674408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c2e694-96c4-4b26-92c9-34641ae8c2d6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:21.674408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 645/1292] 10.222.0.105 () {56 vars in 1121 bytes} [Wed Aug 24 04:45:21 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:23.561576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab0c67-3737-4ddc-b31a-0c4a82208bf2 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:23.562114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab0c67-3737-4ddc-b31a-0c4a82208bf2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:23.562114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab0c67-3737-4ddc-b31a-0c4a82208bf2 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:23.563488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 648/1293] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:45:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:45:23.568897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1f9b7-4895-4699-844b-649dfb1e4d0d None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:23.569306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1f9b7-4895-4699-844b-649dfb1e4d0d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:23.569451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1f9b7-4895-4699-844b-649dfb1e4d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:23.587828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-d0f1f9b7-4895-4699-844b-649dfb1e4d0d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:45:23.589004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d0f1f9b7-4895-4699-844b-649dfb1e4d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:45:23.618952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:23.656283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 646/1294] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:45:23 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:45:23.928901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab0c67-3737-4ddc-b31a-0c4a82208bf2 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:23.959827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:24.030660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab0c67-3737-4ddc-b31a-0c4a82208bf2 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:24.063687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87592248-5dca-4b9a-ab10-7bee322f524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:24.064308 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87592248-5dca-4b9a-ab10-7bee322f524e None admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:24.064586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87592248-5dca-4b9a-ab10-7bee322f524e None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:24.064857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87592248-5dca-4b9a-ab10-7bee322f524e None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:24.065515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87592248-5dca-4b9a-ab10-7bee322f524e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:24.066865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87592248-5dca-4b9a-ab10-7bee322f524e None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:24.167700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 649/1295] 10.222.0.105 () {60 vars in 1146 bytes} [Wed Aug 24 04:45:23 2022] POST /identity/v3/services => generated 247 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:45:26.136263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae981fdc-6ae5-4ef0-a360-c54f4c91b24d None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:26.136792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae981fdc-6ae5-4ef0-a360-c54f4c91b24d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:26.136792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae981fdc-6ae5-4ef0-a360-c54f4c91b24d None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:26.138714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 647/1296] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:45:26 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:45:26.144053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577e46d-382b-46aa-90bc-e6b20d877b4e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:26.144481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577e46d-382b-46aa-90bc-e6b20d877b4e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:26.144577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577e46d-382b-46aa-90bc-e6b20d877b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:26.162331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c577e46d-382b-46aa-90bc-e6b20d877b4e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:45:26.163552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c577e46d-382b-46aa-90bc-e6b20d877b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:45:26.235963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 650/1297] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:45:26 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:45:26.507252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae981fdc-6ae5-4ef0-a360-c54f4c91b24d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:26.604524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae981fdc-6ae5-4ef0-a360-c54f4c91b24d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:26.637405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119ac62-302b-4552-9bd4-73e757854493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:26.638088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119ac62-302b-4552-9bd4-73e757854493 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:26.638356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119ac62-302b-4552-9bd4-73e757854493 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:26.638619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119ac62-302b-4552-9bd4-73e757854493 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:26.639218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119ac62-302b-4552-9bd4-73e757854493 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:26.640561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2119ac62-302b-4552-9bd4-73e757854493 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-2119ac62-302b-4552-9bd4-73e757854493 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:26.649000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:26.655290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:26.656324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 648/1298] 10.222.0.105 () {56 vars in 1122 bytes} [Wed Aug 24 04:45:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:26.657158 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577e46d-382b-46aa-90bc-e6b20d877b4e None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:26.674832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:26.751219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577e46d-382b-46aa-90bc-e6b20d877b4e None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:26.783835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237366bb-1d39-4dd7-884c-89265c29ee22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:26.784369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237366bb-1d39-4dd7-884c-89265c29ee22 None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:26.784643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237366bb-1d39-4dd7-884c-89265c29ee22 None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:26.784905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237366bb-1d39-4dd7-884c-89265c29ee22 None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:26.785525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237366bb-1d39-4dd7-884c-89265c29ee22 None admin] RBAC: Adding query filter params (name=placement) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:26.785818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237366bb-1d39-4dd7-884c-89265c29ee22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:26.787301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237366bb-1d39-4dd7-884c-89265c29ee22 None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:26.792190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 651/1299] 10.222.0.105 () {56 vars in 1121 bytes} [Wed Aug 24 04:45:26 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 04:45:26.799393 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119ac62-302b-4552-9bd4-73e757854493 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:26.830069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:26.896002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119ac62-302b-4552-9bd4-73e757854493 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:26.930886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5eac7-dcf6-41e5-8637-bd13c69490e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:26.931507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5eac7-dcf6-41e5-8637-bd13c69490e6 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:26.931787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5eac7-dcf6-41e5-8637-bd13c69490e6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:26.932057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5eac7-dcf6-41e5-8637-bd13c69490e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:26.932745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5eac7-dcf6-41e5-8637-bd13c69490e6 None admin] RBAC: Adding query filter params (interface=public, service_id=f330671f92d74fdd8a07a77b00ac536c, region_id=RegionOne) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:26.933048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5eac7-dcf6-41e5-8637-bd13c69490e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:26.934387 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5eac7-dcf6-41e5-8637-bd13c69490e6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:26.939218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 649/1300] 10.222.0.105 () {56 vars in 1256 bytes} [Wed Aug 24 04:45:26 2022] GET /identity/v3/endpoints?service_id=f330671f92d74fdd8a07a77b00ac536c&interface=public®ion_id=RegionOne => generated 195 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:26.944926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237366bb-1d39-4dd7-884c-89265c29ee22 None admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:27.024902 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:27.043305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237366bb-1d39-4dd7-884c-89265c29ee22 None admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:27.076662 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221e25d-5cbf-42e4-aee8-614d0ca00ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:27.077472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221e25d-5cbf-42e4-aee8-614d0ca00ddf None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:27.077762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221e25d-5cbf-42e4-aee8-614d0ca00ddf None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:27.078451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221e25d-5cbf-42e4-aee8-614d0ca00ddf None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:27.079076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221e25d-5cbf-42e4-aee8-614d0ca00ddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:27.080434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221e25d-5cbf-42e4-aee8-614d0ca00ddf None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:27.087348 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 652/1301] 10.222.0.105 () {56 vars in 1092 bytes} [Wed Aug 24 04:45:26 2022] GET /identity/v3/services => generated 2201 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:45:29.002214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7182bb-fb68-4ebf-b828-3a579336129c None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:29.002863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7182bb-fb68-4ebf-b828-3a579336129c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:29.002863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7182bb-fb68-4ebf-b828-3a579336129c None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:29.004136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 650/1302] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:45:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:45:29.009450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9492af6-e9ad-40ca-9458-be1d315826f7 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:29.009871 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9492af6-e9ad-40ca-9458-be1d315826f7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:29.010002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9492af6-e9ad-40ca-9458-be1d315826f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:29.028353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b9492af6-e9ad-40ca-9458-be1d315826f7 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:45:29.030327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b9492af6-e9ad-40ca-9458-be1d315826f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:45:29.096699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 653/1303] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:45:29 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 04:45:29.363242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7182bb-fb68-4ebf-b828-3a579336129c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:29.457388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:29.464095 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7182bb-fb68-4ebf-b828-3a579336129c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:29.496753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e8bcf-e8a7-4803-9144-23c7bf580877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:29.497364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e8bcf-e8a7-4803-9144-23c7bf580877 None admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:29.498373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e8bcf-e8a7-4803-9144-23c7bf580877 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:29.498840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e8bcf-e8a7-4803-9144-23c7bf580877 None admin] PATH_INFO: `/v3/services/placement` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:29.499448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e8bcf-e8a7-4803-9144-23c7bf580877 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:29.500792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59e8bcf-e8a7-4803-9144-23c7bf580877 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application [None req-f59e8bcf-e8a7-4803-9144-23c7bf580877 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 04:45:29.509168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 04:45:29.510490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application with Lock( Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self._enter() Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application return self.creator() Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application created_value = creator( Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: ERROR keystone.server.flask.application Aug 24 04:45:29.511512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 651/1304] 10.222.0.105 () {56 vars in 1122 bytes} [Wed Aug 24 04:45:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 04:45:29.516373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9492af6-e9ad-40ca-9458-be1d315826f7 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:29.591131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:29.614652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9492af6-e9ad-40ca-9458-be1d315826f7 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:29.653011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b880ae-cec6-48c2-8981-9fee1200128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:29.653612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b880ae-cec6-48c2-8981-9fee1200128d None admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:29.653889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b880ae-cec6-48c2-8981-9fee1200128d None admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:29.657040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b880ae-cec6-48c2-8981-9fee1200128d None admin] PATH_INFO: `/v3/services` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:29.657040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b880ae-cec6-48c2-8981-9fee1200128d None admin] RBAC: Adding query filter params (name=placement) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:45:29.657040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b880ae-cec6-48c2-8981-9fee1200128d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:29.657315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b880ae-cec6-48c2-8981-9fee1200128d None admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:29.662874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 654/1305] 10.222.0.105 () {56 vars in 1121 bytes} [Wed Aug 24 04:45:29 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:45:29.672122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e8bcf-e8a7-4803-9144-23c7bf580877 None admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:45:29.774427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59e8bcf-e8a7-4803-9144-23c7bf580877 None admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:45:29.813645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fff90d-3c21-409e-84ed-4b9dc120f1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:45:29.814255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff90d-3c21-409e-84ed-4b9dc120f1b6 None admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:45:29.814533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff90d-3c21-409e-84ed-4b9dc120f1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:45:29.814798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fff90d-3c21-409e-84ed-4b9dc120f1b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:45:29.815480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fff90d-3c21-409e-84ed-4b9dc120f1b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:45:29.816846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fff90d-3c21-409e-84ed-4b9dc120f1b6 None admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:45:29.896215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 652/1306] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:45:29 2022] POST /identity/v3/endpoints => generated 333 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:45:39.659666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=30 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:45:39.891717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:08.078870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2059942-410a-4e99-aa42-bf634a05517a None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:08.079376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2059942-410a-4e99-aa42-bf634a05517a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:08.079454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2059942-410a-4e99-aa42-bf634a05517a None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:08.086345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b06e96-9ff3-4ec4-a984-8e6c44af663c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:08.086738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b06e96-9ff3-4ec4-a984-8e6c44af663c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:08.086898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b06e96-9ff3-4ec4-a984-8e6c44af663c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:08.091557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 655/1307] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:08 2022] GET /identity => generated 271 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:08.164467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-65b06e96-9ff3-4ec4-a984-8e6c44af663c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:08.165714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-65b06e96-9ff3-4ec4-a984-8e6c44af663c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:08.281178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 653/1308] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:51:08.576108 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb5f8f-ab13-4ccf-8ddf-c5f951b69424 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:08.576439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb5f8f-ab13-4ccf-8ddf-c5f951b69424 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:08.576718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eb5f8f-ab13-4ccf-8ddf-c5f951b69424 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:08.578654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 656/1309] 10.222.0.105 () {54 vars in 970 bytes} [Wed Aug 24 04:51:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:08.584902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f275d44-b7dc-4496-ad05-d4e282c3ef91 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:08.585280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f275d44-b7dc-4496-ad05-d4e282c3ef91 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:08.585626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f275d44-b7dc-4496-ad05-d4e282c3ef91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:08.670641 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-3f275d44-b7dc-4496-ad05-d4e282c3ef91 None None] MFA Rules not processed for user `44d2b4015ac74b8b8d0a674419f34cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:08.672219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3f275d44-b7dc-4496-ad05-d4e282c3ef91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:08.697007 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:08.762357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 654/1310] 10.222.0.105 () {58 vars in 1067 bytes} [Wed Aug 24 04:51:08 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:08.771821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb5f8f-ab13-4ccf-8ddf-c5f951b69424 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:08.773725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e0eb5f8f-ab13-4ccf-8ddf-c5f951b69424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:08.902062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eb5f8f-ab13-4ccf-8ddf-c5f951b69424 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:08.980970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b66ca-53c9-4241-a6bd-03dc05f17bbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:08.981490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b66ca-53c9-4241-a6bd-03dc05f17bbc service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:08.981768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b66ca-53c9-4241-a6bd-03dc05f17bbc service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:08.982038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b66ca-53c9-4241-a6bd-03dc05f17bbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:08.983671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f31b66ca-53c9-4241-a6bd-03dc05f17bbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:09.041338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:09.098268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b66ca-53c9-4241-a6bd-03dc05f17bbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:09.099661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b66ca-53c9-4241-a6bd-03dc05f17bbc service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:09.119920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 657/1311] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:11.305619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb27566-78f2-41b2-8efc-c668a8c88938 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:11.306192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb27566-78f2-41b2-8efc-c668a8c88938 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:11.306192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb27566-78f2-41b2-8efc-c668a8c88938 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:11.307677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 655/1312] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:11.313463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fb12f-101d-4da8-8153-6f9b2d32e409 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:11.313891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fb12f-101d-4da8-8153-6f9b2d32e409 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:11.314066 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695fb12f-101d-4da8-8153-6f9b2d32e409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:11.402824 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-695fb12f-101d-4da8-8153-6f9b2d32e409 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:11.403947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-695fb12f-101d-4da8-8153-6f9b2d32e409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:11.510627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 658/1313] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:11.793700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb27566-78f2-41b2-8efc-c668a8c88938 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:11.795298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-dcb27566-78f2-41b2-8efc-c668a8c88938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:11.898738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb27566-78f2-41b2-8efc-c668a8c88938 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:11.968976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:11.971572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37a2a6-4a37-4354-8a2e-9c1e3fc7df4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:11.972180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37a2a6-4a37-4354-8a2e-9c1e3fc7df4c service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:11.972456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37a2a6-4a37-4354-8a2e-9c1e3fc7df4c service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:11.972727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37a2a6-4a37-4354-8a2e-9c1e3fc7df4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:11.974321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-bd37a2a6-4a37-4354-8a2e-9c1e3fc7df4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:12.082652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37a2a6-4a37-4354-8a2e-9c1e3fc7df4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:12.084111 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37a2a6-4a37-4354-8a2e-9c1e3fc7df4c service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:12.102184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 656/1314] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:14.523192 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c2d7c-93e4-4b9f-9311-dda1a2ca15f6 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:14.523724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c2d7c-93e4-4b9f-9311-dda1a2ca15f6 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:14.523724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c2d7c-93e4-4b9f-9311-dda1a2ca15f6 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:14.524999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 659/1315] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:14.530683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fff1c2-15c1-4fc0-bcaf-5341fab5d737 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:14.531247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fff1c2-15c1-4fc0-bcaf-5341fab5d737 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:14.531247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fff1c2-15c1-4fc0-bcaf-5341fab5d737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:14.596637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-05fff1c2-15c1-4fc0-bcaf-5341fab5d737 None None] MFA Rules not processed for user `5109071b2f5d4a8fbd1639a297f22e81`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:14.597854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-05fff1c2-15c1-4fc0-bcaf-5341fab5d737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:14.647826 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:14.694073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 657/1316] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:14 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:14.993018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515f9e0-f46f-4d47-a2cf-4bdf7ee4d8ae None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:14.993304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515f9e0-f46f-4d47-a2cf-4bdf7ee4d8ae None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:14.993582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515f9e0-f46f-4d47-a2cf-4bdf7ee4d8ae None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:14.994984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 660/1317] 10.222.0.105 () {54 vars in 970 bytes} [Wed Aug 24 04:51:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:15.000847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87d385-2508-4894-a6f7-35e0cf36a528 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:15.001161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87d385-2508-4894-a6f7-35e0cf36a528 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:15.001434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87d385-2508-4894-a6f7-35e0cf36a528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:15.075545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7b87d385-2508-4894-a6f7-35e0cf36a528 None None] MFA Rules not processed for user `44d2b4015ac74b8b8d0a674419f34cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:15.076760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7b87d385-2508-4894-a6f7-35e0cf36a528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:15.150444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 658/1318] 10.222.0.105 () {58 vars in 1067 bytes} [Wed Aug 24 04:51:14 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:51:15.160177 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515f9e0-f46f-4d47-a2cf-4bdf7ee4d8ae None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:15.161709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a515f9e0-f46f-4d47-a2cf-4bdf7ee4d8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:15.172214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:15.266068 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515f9e0-f46f-4d47-a2cf-4bdf7ee4d8ae None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:15.338511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92e79f-74f8-4c20-8fa8-0c09c80cd056 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:15.339143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92e79f-74f8-4c20-8fa8-0c09c80cd056 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:15.339421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92e79f-74f8-4c20-8fa8-0c09c80cd056 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:15.339692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92e79f-74f8-4c20-8fa8-0c09c80cd056 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:15.341291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fe92e79f-74f8-4c20-8fa8-0c09c80cd056 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:15.408372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:15.454718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92e79f-74f8-4c20-8fa8-0c09c80cd056 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:15.456178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe92e79f-74f8-4c20-8fa8-0c09c80cd056 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:15.474630 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 661/1319] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:15 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:18.512861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da358126-e69f-47c3-a6b0-cb1c8f7bdf79 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:18.513388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da358126-e69f-47c3-a6b0-cb1c8f7bdf79 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:18.513388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da358126-e69f-47c3-a6b0-cb1c8f7bdf79 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:18.514954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 659/1320] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:18.520095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f7980-0ddc-4e4d-b095-91c8696d0bb6 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:18.520514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f7980-0ddc-4e4d-b095-91c8696d0bb6 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:18.520628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f7980-0ddc-4e4d-b095-91c8696d0bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:18.585079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-1c4f7980-0ddc-4e4d-b095-91c8696d0bb6 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:18.586284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1c4f7980-0ddc-4e4d-b095-91c8696d0bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:18.681246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 662/1321] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:18.972912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da358126-e69f-47c3-a6b0-cb1c8f7bdf79 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:18.974488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-da358126-e69f-47c3-a6b0-cb1c8f7bdf79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:19.024214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:19.077540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da358126-e69f-47c3-a6b0-cb1c8f7bdf79 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:19.148090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40c2cd-f829-4d73-a8ef-6679e97c6308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:19.148681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40c2cd-f829-4d73-a8ef-6679e97c6308 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:19.148954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40c2cd-f829-4d73-a8ef-6679e97c6308 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:19.149220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40c2cd-f829-4d73-a8ef-6679e97c6308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:19.150790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4f40c2cd-f829-4d73-a8ef-6679e97c6308 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:19.245705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:19.258557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40c2cd-f829-4d73-a8ef-6679e97c6308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:19.260062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40c2cd-f829-4d73-a8ef-6679e97c6308 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:19.278139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 660/1322] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:51:22.164511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8701798-93bd-49ec-8ae7-2f7b37becc66 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:22.165051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8701798-93bd-49ec-8ae7-2f7b37becc66 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:22.165051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8701798-93bd-49ec-8ae7-2f7b37becc66 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:22.166387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 663/1323] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:22.173309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ab11-0183-4104-9ab9-14b017ea90b4 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:22.173309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ab11-0183-4104-9ab9-14b017ea90b4 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:22.173309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9ab11-0183-4104-9ab9-14b017ea90b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:22.241909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-22a9ab11-0183-4104-9ab9-14b017ea90b4 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:22.243133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-22a9ab11-0183-4104-9ab9-14b017ea90b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:22.344425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 661/1324] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:22.673808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8701798-93bd-49ec-8ae7-2f7b37becc66 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:22.675375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a8701798-93bd-49ec-8ae7-2f7b37becc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:22.708367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:22.837134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8701798-93bd-49ec-8ae7-2f7b37becc66 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:22.920403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d316cf2-a79e-4ef8-b9f7-721f99c0a74e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:22.920937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d316cf2-a79e-4ef8-b9f7-721f99c0a74e service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:22.921218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d316cf2-a79e-4ef8-b9f7-721f99c0a74e service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:22.921488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d316cf2-a79e-4ef8-b9f7-721f99c0a74e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:22.923152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6d316cf2-a79e-4ef8-b9f7-721f99c0a74e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:23.017806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:23.058778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d316cf2-a79e-4ef8-b9f7-721f99c0a74e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:23.060266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d316cf2-a79e-4ef8-b9f7-721f99c0a74e service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:23.079875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 664/1325] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:25.089772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da4057-ff48-4439-9ad5-7d169640a59c None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:25.090274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da4057-ff48-4439-9ad5-7d169640a59c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:25.090274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da4057-ff48-4439-9ad5-7d169640a59c None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:25.092114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 662/1326] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:25.097607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9aa8e-8b8e-4939-8959-8782d42f0493 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:25.098053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9aa8e-8b8e-4939-8959-8782d42f0493 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:25.098230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9aa8e-8b8e-4939-8959-8782d42f0493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:25.164238 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-69c9aa8e-8b8e-4939-8959-8782d42f0493 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:25.165478 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-69c9aa8e-8b8e-4939-8959-8782d42f0493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:25.265558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 665/1327] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:25.563072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da4057-ff48-4439-9ad5-7d169640a59c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:25.564749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-74da4057-ff48-4439-9ad5-7d169640a59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:25.640585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:25.690553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da4057-ff48-4439-9ad5-7d169640a59c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:25.799295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96ba89-e3c7-4638-bd67-652b88207159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:25.800088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96ba89-e3c7-4638-bd67-652b88207159 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:25.800088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96ba89-e3c7-4638-bd67-652b88207159 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:25.800360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96ba89-e3c7-4638-bd67-652b88207159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:25.802467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8b96ba89-e3c7-4638-bd67-652b88207159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:25.938415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:25.941112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96ba89-e3c7-4638-bd67-652b88207159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:25.942616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96ba89-e3c7-4638-bd67-652b88207159 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:25.961895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 663/1328] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:51:28.017076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cafd4b-8d61-4195-ae94-0b3a55818cad None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:28.017591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cafd4b-8d61-4195-ae94-0b3a55818cad None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:28.017712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cafd4b-8d61-4195-ae94-0b3a55818cad None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:28.018977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 666/1329] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:28.024543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05577e7a-1ad9-4071-94dc-ce4c54691e45 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:28.024977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05577e7a-1ad9-4071-94dc-ce4c54691e45 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:28.025094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05577e7a-1ad9-4071-94dc-ce4c54691e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:28.099521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-05577e7a-1ad9-4071-94dc-ce4c54691e45 None None] MFA Rules not processed for user `5109071b2f5d4a8fbd1639a297f22e81`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:28.100768 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-05577e7a-1ad9-4071-94dc-ce4c54691e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:28.201425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 664/1330] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:28 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:28.496715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cafd4b-8d61-4195-ae94-0b3a55818cad None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:28.498509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-58cafd4b-8d61-4195-ae94-0b3a55818cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:28.533681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:28.617312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cafd4b-8d61-4195-ae94-0b3a55818cad None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:28.690188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ad221-5419-44bc-a24d-40381af1b4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:28.690790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ad221-5419-44bc-a24d-40381af1b4bb service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:28.691095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ad221-5419-44bc-a24d-40381af1b4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:28.691384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ad221-5419-44bc-a24d-40381af1b4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:28.692988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5e0ad221-5419-44bc-a24d-40381af1b4bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:28.769876 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:28.801371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ad221-5419-44bc-a24d-40381af1b4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:28.802789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ad221-5419-44bc-a24d-40381af1b4bb service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:28.821602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 667/1331] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:28 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:31.182389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54364f9d-36db-4336-b0b3-3999733f495f None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:31.182947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54364f9d-36db-4336-b0b3-3999733f495f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:31.182947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54364f9d-36db-4336-b0b3-3999733f495f None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:31.184274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 665/1332] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:31.189720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca271ce-b82a-4240-9179-6f8644d7de1d None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:31.190148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca271ce-b82a-4240-9179-6f8644d7de1d None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:31.190343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca271ce-b82a-4240-9179-6f8644d7de1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:31.254713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-0ca271ce-b82a-4240-9179-6f8644d7de1d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:31.255938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0ca271ce-b82a-4240-9179-6f8644d7de1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:31.350853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 668/1333] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:31.635792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54364f9d-36db-4336-b0b3-3999733f495f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:31.637335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-54364f9d-36db-4336-b0b3-3999733f495f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:31.707232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:31.740278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54364f9d-36db-4336-b0b3-3999733f495f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:31.811765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df31ed-f773-443f-b9c4-2331ab664a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:31.812364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df31ed-f773-443f-b9c4-2331ab664a9f service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:31.812642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df31ed-f773-443f-b9c4-2331ab664a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:31.812909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df31ed-f773-443f-b9c4-2331ab664a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:31.814488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-34df31ed-f773-443f-b9c4-2331ab664a9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:31.921519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df31ed-f773-443f-b9c4-2331ab664a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:31.922938 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df31ed-f773-443f-b9c4-2331ab664a9f service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:31.933008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:31.941979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 666/1334] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:33.888307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde207f8-3ccb-468b-840d-552ce855b30b None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:33.888843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde207f8-3ccb-468b-840d-552ce855b30b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:33.888843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde207f8-3ccb-468b-840d-552ce855b30b None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:33.891274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 669/1335] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:33 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:33.896613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e1b59-0081-4cf8-a9bb-4eb2425ba2c3 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:33.897129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e1b59-0081-4cf8-a9bb-4eb2425ba2c3 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:33.897227 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e1b59-0081-4cf8-a9bb-4eb2425ba2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:33.961814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ab1e1b59-0081-4cf8-a9bb-4eb2425ba2c3 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:33.963027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ab1e1b59-0081-4cf8-a9bb-4eb2425ba2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:34.057908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 667/1336] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:34.346641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde207f8-3ccb-468b-840d-552ce855b30b None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:34.348227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cde207f8-3ccb-468b-840d-552ce855b30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:34.388904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:34.452476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde207f8-3ccb-468b-840d-552ce855b30b None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:34.523193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb2b90-103c-44e2-b0cc-4d20fc78ade8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:34.523790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb2b90-103c-44e2-b0cc-4d20fc78ade8 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:34.524068 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb2b90-103c-44e2-b0cc-4d20fc78ade8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:34.524338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb2b90-103c-44e2-b0cc-4d20fc78ade8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:34.525924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-30fb2b90-103c-44e2-b0cc-4d20fc78ade8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:34.612437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:34.638176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb2b90-103c-44e2-b0cc-4d20fc78ade8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:34.639733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb2b90-103c-44e2-b0cc-4d20fc78ade8 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:34.658529 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 670/1337] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:37.405999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99cd4-32f4-42fb-ba73-4fcfb458ff1e None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:37.406584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99cd4-32f4-42fb-ba73-4fcfb458ff1e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:37.406584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d99cd4-32f4-42fb-ba73-4fcfb458ff1e None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:37.407913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 668/1338] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:37.413423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b5c7f-9eaa-47fb-b87c-8a4c9f68c4ec None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:37.413838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b5c7f-9eaa-47fb-b87c-8a4c9f68c4ec None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:37.413936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b5c7f-9eaa-47fb-b87c-8a4c9f68c4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:37.478848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-a82b5c7f-9eaa-47fb-b87c-8a4c9f68c4ec None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:37.480082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a82b5c7f-9eaa-47fb-b87c-8a4c9f68c4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:37.576260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 671/1339] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:37.856897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d99cd4-32f4-42fb-ba73-4fcfb458ff1e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:37.858433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-27d99cd4-32f4-42fb-ba73-4fcfb458ff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:37.934696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:37.961675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d99cd4-32f4-42fb-ba73-4fcfb458ff1e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:38.034735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da89b4-b77f-42c1-84ae-36e89850bb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:38.035367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da89b4-b77f-42c1-84ae-36e89850bb31 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:38.035646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da89b4-b77f-42c1-84ae-36e89850bb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:38.035914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da89b4-b77f-42c1-84ae-36e89850bb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:38.037550 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b0da89b4-b77f-42c1-84ae-36e89850bb31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:38.145249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da89b4-b77f-42c1-84ae-36e89850bb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:38.146603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da89b4-b77f-42c1-84ae-36e89850bb31 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:38.164250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 669/1340] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:42.284377 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1580db-3a70-4041-88d1-3903f988cddc None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:42.284972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1580db-3a70-4041-88d1-3903f988cddc None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:42.284972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1580db-3a70-4041-88d1-3903f988cddc None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:42.286269 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 672/1341] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:42 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:42.291501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2ffe-e88a-44d2-aa39-ed8aa52e3b84 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:42.291936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2ffe-e88a-44d2-aa39-ed8aa52e3b84 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:42.292041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc2ffe-e88a-44d2-aa39-ed8aa52e3b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:42.297553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:42.360363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-56dc2ffe-e88a-44d2-aa39-ed8aa52e3b84 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:42.361589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-56dc2ffe-e88a-44d2-aa39-ed8aa52e3b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:42.467172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 670/1342] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:42.746057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1580db-3a70-4041-88d1-3903f988cddc None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:42.748369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0e1580db-3a70-4041-88d1-3903f988cddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:42.799075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:42.856460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1580db-3a70-4041-88d1-3903f988cddc None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:42.930014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397f160-504f-499c-a6ed-75cb43184331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:42.930622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397f160-504f-499c-a6ed-75cb43184331 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:42.930901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397f160-504f-499c-a6ed-75cb43184331 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:42.931202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397f160-504f-499c-a6ed-75cb43184331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:42.932823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2397f160-504f-499c-a6ed-75cb43184331 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:43.027456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:43.043548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397f160-504f-499c-a6ed-75cb43184331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:43.044997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397f160-504f-499c-a6ed-75cb43184331 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:43.064573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 673/1343] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:45.959377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3f1d5-6035-4d51-80db-f9b32819bbb2 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:45.959927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3f1d5-6035-4d51-80db-f9b32819bbb2 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:45.959927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3f1d5-6035-4d51-80db-f9b32819bbb2 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:45.961407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 671/1344] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:45.966920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212594a2-2f13-45b8-911b-6e129ca30561 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:45.967435 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212594a2-2f13-45b8-911b-6e129ca30561 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:45.967519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212594a2-2f13-45b8-911b-6e129ca30561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:46.035433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-212594a2-2f13-45b8-911b-6e129ca30561 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:46.036514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-212594a2-2f13-45b8-911b-6e129ca30561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:46.135368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 674/1345] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:46.422448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3f1d5-6035-4d51-80db-f9b32819bbb2 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:46.424042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-03f3f1d5-6035-4d51-80db-f9b32819bbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:46.513833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:46.530191 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3f1d5-6035-4d51-80db-f9b32819bbb2 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:46.605137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32608311-075b-4a42-a4db-ca8849bcfc5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:46.605932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32608311-075b-4a42-a4db-ca8849bcfc5b service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:46.606212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32608311-075b-4a42-a4db-ca8849bcfc5b service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:46.606481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32608311-075b-4a42-a4db-ca8849bcfc5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:46.608197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-32608311-075b-4a42-a4db-ca8849bcfc5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:46.721101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32608311-075b-4a42-a4db-ca8849bcfc5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:46.722534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32608311-075b-4a42-a4db-ca8849bcfc5b service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:46.743696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 672/1346] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:50.926273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519de202-84c6-49bc-8104-c68edc6dff59 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:50.926769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519de202-84c6-49bc-8104-c68edc6dff59 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:50.926887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519de202-84c6-49bc-8104-c68edc6dff59 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:50.928184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 675/1347] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:51:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:50.934815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c33b5-35bc-474d-9bc7-2a4c258baab8 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:50.937341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c33b5-35bc-474d-9bc7-2a4c258baab8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:50.937341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c33b5-35bc-474d-9bc7-2a4c258baab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:50.952421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:51.006967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-f49c33b5-35bc-474d-9bc7-2a4c258baab8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:51.008373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f49c33b5-35bc-474d-9bc7-2a4c258baab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:51.113181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 673/1348] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:51:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:51:51.400520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519de202-84c6-49bc-8104-c68edc6dff59 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:51:51.402759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-519de202-84c6-49bc-8104-c68edc6dff59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:51.462589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:51.512977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519de202-84c6-49bc-8104-c68edc6dff59 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:51:51.599384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db6cca-3e78-4bed-934a-6458098bcbce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:51:51.600019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db6cca-3e78-4bed-934a-6458098bcbce service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:51.600298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db6cca-3e78-4bed-934a-6458098bcbce service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:51.600564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db6cca-3e78-4bed-934a-6458098bcbce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:51.602201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d8db6cca-3e78-4bed-934a-6458098bcbce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:51.710753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:51:51.714038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db6cca-3e78-4bed-934a-6458098bcbce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:51:51.715642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db6cca-3e78-4bed-934a-6458098bcbce service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:51:51.733198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 676/1349] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:51:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:51:56.645495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00277304-0b3f-45dd-9491-99ba911210b8 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:56.645995 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00277304-0b3f-45dd-9491-99ba911210b8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:56.646510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00277304-0b3f-45dd-9491-99ba911210b8 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:56.648172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 674/1350] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 04:51:56 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:51:56.657069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d34dd0-5cdd-416b-9983-f720f95fbaec None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:51:56.657523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d34dd0-5cdd-416b-9983-f720f95fbaec None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:51:56.657606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d34dd0-5cdd-416b-9983-f720f95fbaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:51:56.767544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-71d34dd0-5cdd-416b-9983-f720f95fbaec None None] MFA Rules not processed for user `ad8825c294534a7c88877b48b5f8f07f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:51:56.779077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-71d34dd0-5cdd-416b-9983-f720f95fbaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:51:56.939197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 677/1351] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 04:51:56 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:01.111212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=59 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:01.273637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763618d-db99-4b14-8b05-06a2c998e663 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:01.274008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763618d-db99-4b14-8b05-06a2c998e663 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:01.274232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763618d-db99-4b14-8b05-06a2c998e663 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:01.275744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 675/1352] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 04:52:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:01.281400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6d9f6-fd8d-4ed1-ac79-7061a27537b3 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:01.282283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6d9f6-fd8d-4ed1-ac79-7061a27537b3 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:01.282795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6d9f6-fd8d-4ed1-ac79-7061a27537b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:01.344429 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:01.368802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-62b6d9f6-fd8d-4ed1-ac79-7061a27537b3 None None] MFA Rules not processed for user `ad8825c294534a7c88877b48b5f8f07f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:01.371589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-62b6d9f6-fd8d-4ed1-ac79-7061a27537b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:01.497771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 678/1353] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 04:52:01 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:02.817329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52d2c-db75-4283-b307-6bbf995b2beb None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:02.817853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52d2c-db75-4283-b307-6bbf995b2beb None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:02.817996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb52d2c-db75-4283-b307-6bbf995b2beb None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:02.819468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 676/1354] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 04:52:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:02.825383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4436-c089-48f0-b982-d67fe9d5fd85 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:02.825804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4436-c089-48f0-b982-d67fe9d5fd85 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:02.826070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee4436-c089-48f0-b982-d67fe9d5fd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:02.904000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2eee4436-c089-48f0-b982-d67fe9d5fd85 None None] MFA Rules not processed for user `ad8825c294534a7c88877b48b5f8f07f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:02.905213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2eee4436-c089-48f0-b982-d67fe9d5fd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:02.946687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:03.013300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 679/1355] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 04:52:02 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:52:06.267654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc78d7-1f3f-441e-8e3a-44ff7fab2724 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:06.268215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc78d7-1f3f-441e-8e3a-44ff7fab2724 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:06.268215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc78d7-1f3f-441e-8e3a-44ff7fab2724 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:06.269534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 677/1356] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:06.274788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59074da1-d423-4b5b-b989-41cd9d05aedc None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:06.275230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59074da1-d423-4b5b-b989-41cd9d05aedc None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:06.275342 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59074da1-d423-4b5b-b989-41cd9d05aedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:06.357971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-59074da1-d423-4b5b-b989-41cd9d05aedc None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:06.359295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-59074da1-d423-4b5b-b989-41cd9d05aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:06.489585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 680/1357] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:06.796529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ff69a-a100-488a-9c8a-2fda97abd812 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:06.797414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ff69a-a100-488a-9c8a-2fda97abd812 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:06.797981 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ff69a-a100-488a-9c8a-2fda97abd812 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:06.799902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 678/1358] 10.222.0.105 () {54 vars in 967 bytes} [Wed Aug 24 04:52:06 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:06.809644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f1a14-46f3-4cc4-b267-bbc7d9d7b79b None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:06.810287 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f1a14-46f3-4cc4-b267-bbc7d9d7b79b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:06.810884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f1a14-46f3-4cc4-b267-bbc7d9d7b79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:06.831828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:07.022587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-093f1a14-46f3-4cc4-b267-bbc7d9d7b79b None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:07.023816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-093f1a14-46f3-4cc4-b267-bbc7d9d7b79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:07.179447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 681/1359] 10.222.0.105 () {58 vars in 1064 bytes} [Wed Aug 24 04:52:06 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:07.181241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ff69a-a100-488a-9c8a-2fda97abd812 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:07.184531 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5d1ff69a-a100-488a-9c8a-2fda97abd812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:07.344746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ff69a-a100-488a-9c8a-2fda97abd812 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:07.422113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:07.465430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9885f02e-9fa1-4813-9b87-ddebcab3bb5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:07.466122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885f02e-9fa1-4813-9b87-ddebcab3bb5d service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:07.466544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885f02e-9fa1-4813-9b87-ddebcab3bb5d service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:07.466881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9885f02e-9fa1-4813-9b87-ddebcab3bb5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:07.468755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9885f02e-9fa1-4813-9b87-ddebcab3bb5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:07.565353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885f02e-9fa1-4813-9b87-ddebcab3bb5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:07.567046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9885f02e-9fa1-4813-9b87-ddebcab3bb5d service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:07.587809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 679/1360] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:09.753357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2d194-72e8-46f6-91f4-da033b8e4718 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:09.753886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2d194-72e8-46f6-91f4-da033b8e4718 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:09.753886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2d194-72e8-46f6-91f4-da033b8e4718 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:09.755438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 682/1361] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:09.761306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8c92e-f675-4913-a10c-ba5d75e6e5ba None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:09.761750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8c92e-f675-4913-a10c-ba5d75e6e5ba None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:09.761921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8c92e-f675-4913-a10c-ba5d75e6e5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:09.841966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-63d8c92e-f675-4913-a10c-ba5d75e6e5ba None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:09.843213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-63d8c92e-f675-4913-a10c-ba5d75e6e5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:09.895320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:09.962526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 680/1362] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:10.258172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2d194-72e8-46f6-91f4-da033b8e4718 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:10.259854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f9a2d194-72e8-46f6-91f4-da033b8e4718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:10.329161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:10.413931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2d194-72e8-46f6-91f4-da033b8e4718 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:10.545295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d575c5-ff49-4792-8857-79d9678f0406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:10.546195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d575c5-ff49-4792-8857-79d9678f0406 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:10.546624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d575c5-ff49-4792-8857-79d9678f0406 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:10.547439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d575c5-ff49-4792-8857-79d9678f0406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:10.549458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-09d575c5-ff49-4792-8857-79d9678f0406 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:10.580116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:10.662123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d575c5-ff49-4792-8857-79d9678f0406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:10.664223 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d575c5-ff49-4792-8857-79d9678f0406 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:10.686477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 683/1363] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:13.292231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cd688-ab01-4829-b042-1e012f222394 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:13.293149 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cd688-ab01-4829-b042-1e012f222394 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:13.293608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cd688-ab01-4829-b042-1e012f222394 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:13.295169 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 681/1364] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:13 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:13.301074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebdaef-4114-4bce-8277-03d6ade2b6a0 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:13.301511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebdaef-4114-4bce-8277-03d6ade2b6a0 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:13.301806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feebdaef-4114-4bce-8277-03d6ade2b6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:13.377032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-feebdaef-4114-4bce-8277-03d6ade2b6a0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:13.378442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-feebdaef-4114-4bce-8277-03d6ade2b6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:13.480038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:13.483069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 684/1365] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:52:13.776494 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97922a-8a41-45b0-bdbe-89dbcbf6c292 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:13.776923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97922a-8a41-45b0-bdbe-89dbcbf6c292 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:13.777294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97922a-8a41-45b0-bdbe-89dbcbf6c292 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:13.779053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 682/1366] 10.222.0.105 () {54 vars in 967 bytes} [Wed Aug 24 04:52:13 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:13.785063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b76da-1256-4b56-b4cc-fcba53391efd None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:13.787245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b76da-1256-4b56-b4cc-fcba53391efd None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:13.787245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b76da-1256-4b56-b4cc-fcba53391efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:13.868328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-377b76da-1256-4b56-b4cc-fcba53391efd None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:13.869767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-377b76da-1256-4b56-b4cc-fcba53391efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:13.992858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 685/1367] 10.222.0.105 () {58 vars in 1064 bytes} [Wed Aug 24 04:52:13 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:14.001538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97922a-8a41-45b0-bdbe-89dbcbf6c292 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:14.003097 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-1f97922a-8a41-45b0-bdbe-89dbcbf6c292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:14.162564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97922a-8a41-45b0-bdbe-89dbcbf6c292 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:14.194397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:14.289824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd76a5c4-851d-42ea-8061-ec647977f7da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:14.290489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76a5c4-851d-42ea-8061-ec647977f7da service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:14.290769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76a5c4-851d-42ea-8061-ec647977f7da service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:14.291091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd76a5c4-851d-42ea-8061-ec647977f7da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:14.292851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-cd76a5c4-851d-42ea-8061-ec647977f7da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:14.387802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76a5c4-851d-42ea-8061-ec647977f7da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:14.389568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd76a5c4-851d-42ea-8061-ec647977f7da service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:14.408172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 683/1368] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:14.638821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065c703-4c7b-4019-98ad-99f5c0d6e072 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:14.639407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065c703-4c7b-4019-98ad-99f5c0d6e072 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:14.639407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065c703-4c7b-4019-98ad-99f5c0d6e072 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:14.640782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 686/1369] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:14.646430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0872-5f42-48da-ac62-58cb6f12db5a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:14.646712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0872-5f42-48da-ac62-58cb6f12db5a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:14.646983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d0872-5f42-48da-ac62-58cb6f12db5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:14.692011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:14.738619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-819d0872-5f42-48da-ac62-58cb6f12db5a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:14.740385 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-819d0872-5f42-48da-ac62-58cb6f12db5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:14.850861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 684/1370] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:15.445727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb211b-aad5-4b91-93ac-8a7613337e76 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:15.445727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb211b-aad5-4b91-93ac-8a7613337e76 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:15.448341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb211b-aad5-4b91-93ac-8a7613337e76 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:15.448341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 687/1371] 10.222.0.105 () {54 vars in 969 bytes} [Wed Aug 24 04:52:15 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:15.453410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca140-41d6-4764-9974-73b93d45bd89 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:15.453975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca140-41d6-4764-9974-73b93d45bd89 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:15.453975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca140-41d6-4764-9974-73b93d45bd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:15.542156 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b1dca140-41d6-4764-9974-73b93d45bd89 None None] MFA Rules not processed for user `29af87e1e05b4880b9ec96865994b635`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:15.543536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b1dca140-41d6-4764-9974-73b93d45bd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:15.594627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:15.628655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 685/1372] 10.222.0.105 () {58 vars in 1066 bytes} [Wed Aug 24 04:52:15 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:15.637516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb211b-aad5-4b91-93ac-8a7613337e76 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:15.639249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fbdb211b-aad5-4b91-93ac-8a7613337e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:15.693610 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:15.754017 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb211b-aad5-4b91-93ac-8a7613337e76 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:15.837695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80662404-971d-40d9-94b9-8c23a8919bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:15.838268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80662404-971d-40d9-94b9-8c23a8919bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:15.838547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80662404-971d-40d9-94b9-8c23a8919bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:15.838817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80662404-971d-40d9-94b9-8c23a8919bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:15.840449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-80662404-971d-40d9-94b9-8c23a8919bf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:15.939733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:15.956195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80662404-971d-40d9-94b9-8c23a8919bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:15.957699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80662404-971d-40d9-94b9-8c23a8919bf9 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:15.980535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 688/1373] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:16.572051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc36a-4854-4e3c-adf7-3d3971a522e3 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:16.572577 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc36a-4854-4e3c-adf7-3d3971a522e3 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:16.572703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fc36a-4854-4e3c-adf7-3d3971a522e3 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:16.573967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 686/1374] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:16.579540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0ecc5-7b1c-4f4b-ba62-eee72ef98856 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:16.579830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0ecc5-7b1c-4f4b-ba62-eee72ef98856 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:16.580106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0ecc5-7b1c-4f4b-ba62-eee72ef98856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:16.645267 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-63e0ecc5-7b1c-4f4b-ba62-eee72ef98856 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:16.646493 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-63e0ecc5-7b1c-4f4b-ba62-eee72ef98856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:16.746666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 689/1375] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:17.036958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fc36a-4854-4e3c-adf7-3d3971a522e3 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:17.038578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-472fc36a-4854-4e3c-adf7-3d3971a522e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:17.191719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fc36a-4854-4e3c-adf7-3d3971a522e3 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:17.242811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:17.305109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534ea6f-ef15-4d81-a130-1cfdf8e3afb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:17.305711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534ea6f-ef15-4d81-a130-1cfdf8e3afb1 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:17.305983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534ea6f-ef15-4d81-a130-1cfdf8e3afb1 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:17.306250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534ea6f-ef15-4d81-a130-1cfdf8e3afb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:17.308403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8534ea6f-ef15-4d81-a130-1cfdf8e3afb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:17.396841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534ea6f-ef15-4d81-a130-1cfdf8e3afb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:17.398414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534ea6f-ef15-4d81-a130-1cfdf8e3afb1 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:17.418381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 687/1376] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:19.523264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3fccc-650a-4eaf-b369-628fc25509d1 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:19.523685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3fccc-650a-4eaf-b369-628fc25509d1 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:19.523879 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a3fccc-650a-4eaf-b369-628fc25509d1 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:19.525351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 690/1377] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:19.530882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddd913-e0f1-4f01-9849-5b6a502c6d0e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:19.531334 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddd913-e0f1-4f01-9849-5b6a502c6d0e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:19.531505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddd913-e0f1-4f01-9849-5b6a502c6d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:19.615728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-72ddd913-e0f1-4f01-9849-5b6a502c6d0e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:19.616933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-72ddd913-e0f1-4f01-9849-5b6a502c6d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:19.624830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:19.748885 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 688/1378] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:20.052999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3fccc-650a-4eaf-b369-628fc25509d1 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:20.054512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a9a3fccc-650a-4eaf-b369-628fc25509d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:20.121390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:20.217109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a3fccc-650a-4eaf-b369-628fc25509d1 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:20.338413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79572e49-24e4-4280-86b9-64664c8b5d27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:20.339041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79572e49-24e4-4280-86b9-64664c8b5d27 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:20.339321 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79572e49-24e4-4280-86b9-64664c8b5d27 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:20.339591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79572e49-24e4-4280-86b9-64664c8b5d27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:20.341197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-79572e49-24e4-4280-86b9-64664c8b5d27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:20.360778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:20.435781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79572e49-24e4-4280-86b9-64664c8b5d27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:20.437313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79572e49-24e4-4280-86b9-64664c8b5d27 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:20.459209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 691/1379] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:22.532048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f32fe-9557-48f3-921b-331589ff43e2 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:22.532683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f32fe-9557-48f3-921b-331589ff43e2 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:22.532683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f32fe-9557-48f3-921b-331589ff43e2 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:22.538848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 689/1380] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:22 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:22.542380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c3a4-fba7-4678-a1e6-d745bd7bef29 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:22.542943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c3a4-fba7-4678-a1e6-d745bd7bef29 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:22.542943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2c3a4-fba7-4678-a1e6-d745bd7bef29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:22.629768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-4ac2c3a4-fba7-4678-a1e6-d745bd7bef29 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:22.631010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4ac2c3a4-fba7-4678-a1e6-d745bd7bef29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:22.726597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:22.732685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 692/1381] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:23.026465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f32fe-9557-48f3-921b-331589ff43e2 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:23.028282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3f3f32fe-9557-48f3-921b-331589ff43e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:23.183120 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:23.197688 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f32fe-9557-48f3-921b-331589ff43e2 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:23.325844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55beb88c-055f-4787-8dd6-f16e24bdd3de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:23.326636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55beb88c-055f-4787-8dd6-f16e24bdd3de service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:23.327063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55beb88c-055f-4787-8dd6-f16e24bdd3de service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:23.327435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55beb88c-055f-4787-8dd6-f16e24bdd3de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:23.329304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-55beb88c-055f-4787-8dd6-f16e24bdd3de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:23.426431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55beb88c-055f-4787-8dd6-f16e24bdd3de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:23.428053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55beb88c-055f-4787-8dd6-f16e24bdd3de service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:23.438876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:23.447899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 690/1382] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:25.773899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90772f59-a432-4448-b8fc-dbf5eb4893db None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:25.774443 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90772f59-a432-4448-b8fc-dbf5eb4893db None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:25.774652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90772f59-a432-4448-b8fc-dbf5eb4893db None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:25.777530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 693/1383] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:25 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:25.783610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5f2a7-e805-44fc-845a-c37b478ede28 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:25.784025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5f2a7-e805-44fc-845a-c37b478ede28 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:25.784377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad5f2a7-e805-44fc-845a-c37b478ede28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:25.873734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-aad5f2a7-e805-44fc-845a-c37b478ede28 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:25.875340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-aad5f2a7-e805-44fc-845a-c37b478ede28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:25.985478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 691/1384] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:26.278603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90772f59-a432-4448-b8fc-dbf5eb4893db None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:26.280166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-90772f59-a432-4448-b8fc-dbf5eb4893db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:26.443031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90772f59-a432-4448-b8fc-dbf5eb4893db None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:26.511609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:26.560071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9ccac-b827-4193-96e1-1e57564e8622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:26.560880 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9ccac-b827-4193-96e1-1e57564e8622 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:26.561329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9ccac-b827-4193-96e1-1e57564e8622 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:26.561957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9ccac-b827-4193-96e1-1e57564e8622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:26.563603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-11d9ccac-b827-4193-96e1-1e57564e8622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:26.656333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9ccac-b827-4193-96e1-1e57564e8622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:26.657878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9ccac-b827-4193-96e1-1e57564e8622 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:26.676555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 694/1385] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:26 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:28.597177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00b0df-7c95-4a82-a3f5-3820a7af7c49 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:28.597732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00b0df-7c95-4a82-a3f5-3820a7af7c49 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:28.597732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00b0df-7c95-4a82-a3f5-3820a7af7c49 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:28.599094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 692/1386] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:28.605020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3751e24-c9bf-4ddb-be20-17b55a5958b0 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:28.605463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3751e24-c9bf-4ddb-be20-17b55a5958b0 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:28.605563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3751e24-c9bf-4ddb-be20-17b55a5958b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:28.672379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b3751e24-c9bf-4ddb-be20-17b55a5958b0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:28.673867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b3751e24-c9bf-4ddb-be20-17b55a5958b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:28.707244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:28.776248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 695/1387] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:29.069742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00b0df-7c95-4a82-a3f5-3820a7af7c49 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:29.071291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-da00b0df-7c95-4a82-a3f5-3820a7af7c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:29.150205 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:29.223474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00b0df-7c95-4a82-a3f5-3820a7af7c49 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:29.356399 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb81cc8-6b5d-4072-94bf-535612c30afc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:29.357594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb81cc8-6b5d-4072-94bf-535612c30afc service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:29.357594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb81cc8-6b5d-4072-94bf-535612c30afc service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:29.357594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb81cc8-6b5d-4072-94bf-535612c30afc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:29.359352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2cb81cc8-6b5d-4072-94bf-535612c30afc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:29.405020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:29.463825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb81cc8-6b5d-4072-94bf-535612c30afc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:29.465477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb81cc8-6b5d-4072-94bf-535612c30afc service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:29.485490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 693/1388] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:31.564643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679faedc-d9ec-4d6f-aa16-8b728af44e70 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:31.565179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679faedc-d9ec-4d6f-aa16-8b728af44e70 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:31.565179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679faedc-d9ec-4d6f-aa16-8b728af44e70 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:31.566543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 696/1389] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:31.572794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419c152-dbdb-4f2d-88b9-d8422203a7b7 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:31.573239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419c152-dbdb-4f2d-88b9-d8422203a7b7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:31.573362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419c152-dbdb-4f2d-88b9-d8422203a7b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:31.640321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2419c152-dbdb-4f2d-88b9-d8422203a7b7 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:31.641526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2419c152-dbdb-4f2d-88b9-d8422203a7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:31.662437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a21aa3-26e3-424f-8e8c-d003c3741455 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:31.662718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a21aa3-26e3-424f-8e8c-d003c3741455 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:31.662995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a21aa3-26e3-424f-8e8c-d003c3741455 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:31.664362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 697/1390] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:52:31.669520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcca1b-e5cf-4f7c-b484-fd0f2aa66a07 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:31.669809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcca1b-e5cf-4f7c-b484-fd0f2aa66a07 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:31.670089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bcca1b-e5cf-4f7c-b484-fd0f2aa66a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:31.737501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-44bcca1b-e5cf-4f7c-b484-fd0f2aa66a07 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:31.738709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-44bcca1b-e5cf-4f7c-b484-fd0f2aa66a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:31.743128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 694/1391] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:31.841299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 698/1392] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:52:32.044241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419c152-dbdb-4f2d-88b9-d8422203a7b7 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:32.045498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2419c152-dbdb-4f2d-88b9-d8422203a7b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:32.060820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:32.160073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419c152-dbdb-4f2d-88b9-d8422203a7b7 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:32.177400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26732d29-a86b-4aed-8ac0-064e0cd50f47 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:32.177687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26732d29-a86b-4aed-8ac0-064e0cd50f47 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:32.177969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26732d29-a86b-4aed-8ac0-064e0cd50f47 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:32.179343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 699/1393] 10.222.0.105 () {54 vars in 972 bytes} [Wed Aug 24 04:52:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:52:32.184437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5e383-6863-4d24-b930-8abde5404e71 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:32.184717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5e383-6863-4d24-b930-8abde5404e71 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:32.184990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5e383-6863-4d24-b930-8abde5404e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:32.202808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:32.263541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-3dc5e383-6863-4d24-b930-8abde5404e71 None None] MFA Rules not processed for user `f1d62a4c07c4402ea5803f6e68fb1d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:32.264757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3dc5e383-6863-4d24-b930-8abde5404e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:32.280731 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f37399-cb00-441e-b38c-a9e06155c40b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:32.281338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f37399-cb00-441e-b38c-a9e06155c40b service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:32.281615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f37399-cb00-441e-b38c-a9e06155c40b service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:32.281886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f37399-cb00-441e-b38c-a9e06155c40b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:32.283514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-82f37399-cb00-441e-b38c-a9e06155c40b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:32.293183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:32.342147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 700/1394] 10.222.0.105 () {58 vars in 1069 bytes} [Wed Aug 24 04:52:32 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:52:32.349832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5e383-6863-4d24-b930-8abde5404e71 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:32.358048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5e383-6863-4d24-b930-8abde5404e71 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:32.374993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f37399-cb00-441e-b38c-a9e06155c40b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:32.376554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f37399-cb00-441e-b38c-a9e06155c40b service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:32.394379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 695/1395] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:32.433671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f712aa2-fbbe-4f56-b737-8cdc440e30fa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d62a4c07c4402ea5803f6e68fb1d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:32.434275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f712aa2-fbbe-4f56-b737-8cdc440e30fa service glance] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:32.434552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f712aa2-fbbe-4f56-b737-8cdc440e30fa service glance] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:32.434820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f712aa2-fbbe-4f56-b737-8cdc440e30fa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:32.436466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5f712aa2-fbbe-4f56-b737-8cdc440e30fa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:32.467002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:32.549821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f712aa2-fbbe-4f56-b737-8cdc440e30fa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:32.551276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f712aa2-fbbe-4f56-b737-8cdc440e30fa service glance] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:32.569682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 701/1396] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:52:33.544548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f34c3-1b40-4910-8597-1ad2b31283a9 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:33.545365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f34c3-1b40-4910-8597-1ad2b31283a9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:33.545365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f34c3-1b40-4910-8597-1ad2b31283a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:33.613299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c90f34c3-1b40-4910-8597-1ad2b31283a9 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:33.614504 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c90f34c3-1b40-4910-8597-1ad2b31283a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:33.700769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:33.704336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 696/1397] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 04:52:33 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:33.751108 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e75c50-cc83-4339-97fa-5165fc91e881 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:33.751605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e75c50-cc83-4339-97fa-5165fc91e881 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:33.751712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e75c50-cc83-4339-97fa-5165fc91e881 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:33.752991 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 702/1398] 10.222.0.105 () {54 vars in 948 bytes} [Wed Aug 24 04:52:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:33.760528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd538d3f-894f-43fd-8dda-9bf1c0c2d351 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:33.760806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd538d3f-894f-43fd-8dda-9bf1c0c2d351 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:33.761075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd538d3f-894f-43fd-8dda-9bf1c0c2d351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:33.835723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-cd538d3f-894f-43fd-8dda-9bf1c0c2d351 None None] MFA Rules not processed for user `e6a894e4e6f24febbd8397fb8c4c7fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:33.837076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-cd538d3f-894f-43fd-8dda-9bf1c0c2d351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:33.942836 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 697/1399] 10.222.0.105 () {58 vars in 1045 bytes} [Wed Aug 24 04:52:33 2022] POST /identity/v3/auth/tokens => generated 3314 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:33.953294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e75c50-cc83-4339-97fa-5165fc91e881 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:33.955006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f7e75c50-cc83-4339-97fa-5165fc91e881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:34.090586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:34.093150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e75c50-cc83-4339-97fa-5165fc91e881 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:34.199065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad7dccc-0287-4579-ad09-b2240b068660 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:34.199682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7dccc-0287-4579-ad09-b2240b068660 service swift] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:34.199964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7dccc-0287-4579-ad09-b2240b068660 service swift] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:34.200227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7dccc-0287-4579-ad09-b2240b068660 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:34.201897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cad7dccc-0287-4579-ad09-b2240b068660 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:34.285441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7dccc-0287-4579-ad09-b2240b068660 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:34.287043 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad7dccc-0287-4579-ad09-b2240b068660 service swift] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:34.295387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 703/1400] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 04:52:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 04:52:34.521165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a8029-6777-484c-8b50-4146774e77f8 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:34.521373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a8029-6777-484c-8b50-4146774e77f8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:34.521623 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17a8029-6777-484c-8b50-4146774e77f8 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:34.522984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 698/1401] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:34.528668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54ce5c-7c9a-464c-bac9-27494e667390 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:34.528945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54ce5c-7c9a-464c-bac9-27494e667390 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:34.529243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea54ce5c-7c9a-464c-bac9-27494e667390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:34.570430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:34.603771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ea54ce5c-7c9a-464c-bac9-27494e667390 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:34.605256 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ea54ce5c-7c9a-464c-bac9-27494e667390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:34.709682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 704/1402] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:35.047383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a8029-6777-484c-8b50-4146774e77f8 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:35.049711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a17a8029-6777-484c-8b50-4146774e77f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:35.130899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:35.257796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17a8029-6777-484c-8b50-4146774e77f8 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:35.414637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5388857-19ee-4cd8-8161-f2bd6eeee38c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:35.416968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5388857-19ee-4cd8-8161-f2bd6eeee38c service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:35.417254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5388857-19ee-4cd8-8161-f2bd6eeee38c service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:35.417683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5388857-19ee-4cd8-8161-f2bd6eeee38c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:35.421139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f5388857-19ee-4cd8-8161-f2bd6eeee38c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:35.445705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:35.528081 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5388857-19ee-4cd8-8161-f2bd6eeee38c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:35.529684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5388857-19ee-4cd8-8161-f2bd6eeee38c service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:35.551290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 699/1403] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:37.786329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690859e-7543-4ce6-94c8-500fcf7d56ac None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:37.787310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690859e-7543-4ce6-94c8-500fcf7d56ac None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:37.787782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690859e-7543-4ce6-94c8-500fcf7d56ac None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:37.789474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 705/1404] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:37.796185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd1237-0b28-4d28-84c0-ba4f27da9482 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:37.796782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd1237-0b28-4d28-84c0-ba4f27da9482 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:37.796782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd1237-0b28-4d28-84c0-ba4f27da9482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:37.875634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-99bd1237-0b28-4d28-84c0-ba4f27da9482 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:37.876903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-99bd1237-0b28-4d28-84c0-ba4f27da9482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:37.992466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:38.001240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 700/1405] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:38.298788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690859e-7543-4ce6-94c8-500fcf7d56ac None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:38.300591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6690859e-7543-4ce6-94c8-500fcf7d56ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:38.418199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:38.471901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690859e-7543-4ce6-94c8-500fcf7d56ac None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:38.608984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f4a09-fbf0-4d21-81a7-8e4fa25ee140 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:38.609692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f4a09-fbf0-4d21-81a7-8e4fa25ee140 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:38.610040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f4a09-fbf0-4d21-81a7-8e4fa25ee140 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:38.610371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f4a09-fbf0-4d21-81a7-8e4fa25ee140 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:38.612087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-443f4a09-fbf0-4d21-81a7-8e4fa25ee140 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:38.695623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:38.718989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f4a09-fbf0-4d21-81a7-8e4fa25ee140 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:38.720570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f4a09-fbf0-4d21-81a7-8e4fa25ee140 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:38.741544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 706/1406] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:40.874348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27f299-7d7c-46aa-b0c5-0c55fe80fef1 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:40.874909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27f299-7d7c-46aa-b0c5-0c55fe80fef1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:40.874909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27f299-7d7c-46aa-b0c5-0c55fe80fef1 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:40.879836 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 701/1407] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:52:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:40.882342 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97713-09bf-4543-a2e9-bd5a3c3f75d5 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:40.882793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97713-09bf-4543-a2e9-bd5a3c3f75d5 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:40.882890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe97713-09bf-4543-a2e9-bd5a3c3f75d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:40.953709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ffe97713-09bf-4543-a2e9-bd5a3c3f75d5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:40.954948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ffe97713-09bf-4543-a2e9-bd5a3c3f75d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:41.059447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 707/1408] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:52:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:52:41.361541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27f299-7d7c-46aa-b0c5-0c55fe80fef1 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:52:41.363115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3c27f299-7d7c-46aa-b0c5-0c55fe80fef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:41.528721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27f299-7d7c-46aa-b0c5-0c55fe80fef1 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:52:41.602186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:41.653762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe8ee6-69d6-48ff-8d09-68f424d56265 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:52:41.654372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe8ee6-69d6-48ff-8d09-68f424d56265 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:41.654643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe8ee6-69d6-48ff-8d09-68f424d56265 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:41.654908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe8ee6-69d6-48ff-8d09-68f424d56265 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:41.656547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7ffe8ee6-69d6-48ff-8d09-68f424d56265 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:41.761038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe8ee6-69d6-48ff-8d09-68f424d56265 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:52:41.764480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe8ee6-69d6-48ff-8d09-68f424d56265 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:52:41.787418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 702/1409] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:52:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:52:51.056847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=79 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:51.780215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=64 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:58.043905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5186497-d795-4f95-b4d5-b1d1af5a9b5a None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:58.044506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5186497-d795-4f95-b4d5-b1d1af5a9b5a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:58.044506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5186497-d795-4f95-b4d5-b1d1af5a9b5a None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:58.045829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 708/1410] 10.222.0.105 () {54 vars in 923 bytes} [Wed Aug 24 04:52:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:58.051253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddc44c-27c0-412f-aed2-9c84b440e5e6 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:58.051643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddc44c-27c0-412f-aed2-9c84b440e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:58.051827 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ddc44c-27c0-412f-aed2-9c84b440e5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:58.127559 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-65ddc44c-27c0-412f-aed2-9c84b440e5e6 None None] MFA Rules not processed for user `ad8825c294534a7c88877b48b5f8f07f`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:58.128804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-65ddc44c-27c0-412f-aed2-9c84b440e5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:58.234239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 703/1411] 10.222.0.105 () {58 vars in 1020 bytes} [Wed Aug 24 04:52:58 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:52:58.248536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84b339-066d-496e-91dd-463b04241918 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:58.248816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84b339-066d-496e-91dd-463b04241918 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:58.249084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84b339-066d-496e-91dd-463b04241918 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:58.250390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 709/1412] 10.222.0.105 () {54 vars in 923 bytes} [Wed Aug 24 04:52:58 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:52:58.254910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a259a-d26b-4737-ae7f-770e4954eaa6 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:52:58.255239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a259a-d26b-4737-ae7f-770e4954eaa6 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:52:58.255515 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a259a-d26b-4737-ae7f-770e4954eaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:52:58.329999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:52:58.332970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-909a259a-d26b-4737-ae7f-770e4954eaa6 None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:52:58.334148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-909a259a-d26b-4737-ae7f-770e4954eaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:52:58.448240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 704/1413] 10.222.0.105 () {58 vars in 1020 bytes} [Wed Aug 24 04:52:58 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:00.834278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfed14-4147-4438-b900-1fb1b068b119 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:00.834835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfed14-4147-4438-b900-1fb1b068b119 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:00.834835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfed14-4147-4438-b900-1fb1b068b119 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:00.836220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:00.841444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10cd07-4435-4a43-ac04-1c6f63ee5172 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:00.841852 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10cd07-4435-4a43-ac04-1c6f63ee5172 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:00.841998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10cd07-4435-4a43-ac04-1c6f63ee5172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:00.907349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5d10cd07-4435-4a43-ac04-1c6f63ee5172 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:00.908473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5d10cd07-4435-4a43-ac04-1c6f63ee5172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:00.963243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:01.004999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:01.298536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfed14-4147-4438-b900-1fb1b068b119 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:01.300182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-84cfed14-4147-4438-b900-1fb1b068b119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:01.446848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfed14-4147-4438-b900-1fb1b068b119 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:01.519390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:01.563467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfdcd7-0e5b-4957-a292-15d2e56ec37e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:01.564078 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfdcd7-0e5b-4957-a292-15d2e56ec37e service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:01.564360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfdcd7-0e5b-4957-a292-15d2e56ec37e service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:01.564632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfdcd7-0e5b-4957-a292-15d2e56ec37e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:01.566247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0edfdcd7-0e5b-4957-a292-15d2e56ec37e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:01.655552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfdcd7-0e5b-4957-a292-15d2e56ec37e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:01.657125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfdcd7-0e5b-4957-a292-15d2e56ec37e service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:01.676085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 711/1416] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:01 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:02.573337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8d7bf-f313-4452-9707-017e20b30060 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:02.573864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8d7bf-f313-4452-9707-017e20b30060 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:02.573984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8d7bf-f313-4452-9707-017e20b30060 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:02.575352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:53:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:53:02.580373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6dea6-eed6-4076-be15-d80a76df1f9e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:02.580786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6dea6-eed6-4076-be15-d80a76df1f9e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:02.580922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6dea6-eed6-4076-be15-d80a76df1f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:02.655538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-fbb6dea6-eed6-4076-be15-d80a76df1f9e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:02.656745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fbb6dea6-eed6-4076-be15-d80a76df1f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:02.684490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:02.752883 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:53:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:02.903535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8d7bf-f313-4452-9707-017e20b30060 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:02.905030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-81b8d7bf-f313-4452-9707-017e20b30060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:03.047699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8d7bf-f313-4452-9707-017e20b30060 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:03.087817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:03.160660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965a4a7-6469-477b-85d2-8b347739f802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:03.161179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965a4a7-6469-477b-85d2-8b347739f802 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:03.161455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965a4a7-6469-477b-85d2-8b347739f802 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:03.161718 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965a4a7-6469-477b-85d2-8b347739f802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:03.163341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3965a4a7-6469-477b-85d2-8b347739f802 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:03.252929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965a4a7-6469-477b-85d2-8b347739f802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:03.254542 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3965a4a7-6469-477b-85d2-8b347739f802 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:03.272853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 707/1419] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:04.169608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b41d67-d7ea-4c16-8997-13da14247733 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:04.170233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b41d67-d7ea-4c16-8997-13da14247733 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:04.170233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b41d67-d7ea-4c16-8997-13da14247733 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:04.171575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:53:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:04.177039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d742418-cc33-4f65-84fe-deddbe6509ac None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:04.177485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d742418-cc33-4f65-84fe-deddbe6509ac None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:04.177604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d742418-cc33-4f65-84fe-deddbe6509ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:04.232289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:04.253458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-4d742418-cc33-4f65-84fe-deddbe6509ac None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:04.255008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4d742418-cc33-4f65-84fe-deddbe6509ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:04.349885 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:53:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:53:04.504661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b41d67-d7ea-4c16-8997-13da14247733 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:04.506169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-37b41d67-d7ea-4c16-8997-13da14247733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:04.650412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b41d67-d7ea-4c16-8997-13da14247733 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:04.662276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:04.762356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27855f-3de6-4bd5-b4dd-5a2d199afcb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:04.762878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27855f-3de6-4bd5-b4dd-5a2d199afcb4 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:04.763186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27855f-3de6-4bd5-b4dd-5a2d199afcb4 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:04.763456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27855f-3de6-4bd5-b4dd-5a2d199afcb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:04.765051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2b27855f-3de6-4bd5-b4dd-5a2d199afcb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:04.857669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27855f-3de6-4bd5-b4dd-5a2d199afcb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:04.859688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27855f-3de6-4bd5-b4dd-5a2d199afcb4 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:04.878022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 714/1422] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:05.745272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60416c-f98a-45cd-9217-94b2b3e58fad None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:05.745822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60416c-f98a-45cd-9217-94b2b3e58fad None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:05.745940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a60416c-f98a-45cd-9217-94b2b3e58fad None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:05.747236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:53:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 04:53:05.752045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d51527-cd2b-4c3c-a5ec-a43510218250 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:05.752467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d51527-cd2b-4c3c-a5ec-a43510218250 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:05.752591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d51527-cd2b-4c3c-a5ec-a43510218250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:05.772689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:05.830237 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-98d51527-cd2b-4c3c-a5ec-a43510218250 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:05.831366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-98d51527-cd2b-4c3c-a5ec-a43510218250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:05.926771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:53:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:06.081620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60416c-f98a-45cd-9217-94b2b3e58fad None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:06.083097 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0a60416c-f98a-45cd-9217-94b2b3e58fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:06.191895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:06.228360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a60416c-f98a-45cd-9217-94b2b3e58fad None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:06.339715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a66a4-4466-4483-a674-c6ce7ad3b91e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:06.340265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a66a4-4466-4483-a674-c6ce7ad3b91e service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:06.340542 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a66a4-4466-4483-a674-c6ce7ad3b91e service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:06.341449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a66a4-4466-4483-a674-c6ce7ad3b91e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:06.343080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ff1a66a4-4466-4483-a674-c6ce7ad3b91e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:06.419121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:06.432907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a66a4-4466-4483-a674-c6ce7ad3b91e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:06.434555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a66a4-4466-4483-a674-c6ce7ad3b91e service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:06.452570 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 710/1425] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:07.331009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f66f-023e-4fad-9898-eb542e318e14 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:07.331614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f66f-023e-4fad-9898-eb542e318e14 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:07.331614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16f66f-023e-4fad-9898-eb542e318e14 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:07.332929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:53:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:07.338384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff2b47-2fdc-49a9-9dc6-94f118a1c94c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:07.338834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff2b47-2fdc-49a9-9dc6-94f118a1c94c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:07.338940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff2b47-2fdc-49a9-9dc6-94f118a1c94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:07.413175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-18ff2b47-2fdc-49a9-9dc6-94f118a1c94c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:07.414305 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-18ff2b47-2fdc-49a9-9dc6-94f118a1c94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:07.510076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:53:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:07.661972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f66f-023e-4fad-9898-eb542e318e14 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:07.663545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6b16f66f-023e-4fad-9898-eb542e318e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:07.748004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:07.810555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16f66f-023e-4fad-9898-eb542e318e14 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:07.924137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f124d8-bfb3-420d-b5c9-107a2f3f041c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:07.924663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f124d8-bfb3-420d-b5c9-107a2f3f041c service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:07.924937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f124d8-bfb3-420d-b5c9-107a2f3f041c service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:07.925202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f124d8-bfb3-420d-b5c9-107a2f3f041c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:07.926803 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f8f124d8-bfb3-420d-b5c9-107a2f3f041c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:07.974578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:08.015866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f124d8-bfb3-420d-b5c9-107a2f3f041c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:08.017446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f124d8-bfb3-420d-b5c9-107a2f3f041c service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:08.035778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 717/1428] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:08.904131 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abbd4b-7a92-4dfb-84c4-7dcf4422e682 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:08.904682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abbd4b-7a92-4dfb-84c4-7dcf4422e682 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:08.904682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38abbd4b-7a92-4dfb-84c4-7dcf4422e682 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:08.906096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:53:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:08.911037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda487e-ed43-4bd0-95ac-e0bf9a654b18 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:08.911471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda487e-ed43-4bd0-95ac-e0bf9a654b18 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:08.911583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda487e-ed43-4bd0-95ac-e0bf9a654b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:08.988951 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2bda487e-ed43-4bd0-95ac-e0bf9a654b18 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:08.990422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2bda487e-ed43-4bd0-95ac-e0bf9a654b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:09.088442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:53:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:09.239708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abbd4b-7a92-4dfb-84c4-7dcf4422e682 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:09.241224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-38abbd4b-7a92-4dfb-84c4-7dcf4422e682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:09.289779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:09.386197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38abbd4b-7a92-4dfb-84c4-7dcf4422e682 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:09.497912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d1e22-a5f9-411f-94b9-85a4eda15044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:09.498436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d1e22-a5f9-411f-94b9-85a4eda15044 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:09.498715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d1e22-a5f9-411f-94b9-85a4eda15044 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:09.498982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d1e22-a5f9-411f-94b9-85a4eda15044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:09.500677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-398d1e22-a5f9-411f-94b9-85a4eda15044 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:09.507481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:09.589110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d1e22-a5f9-411f-94b9-85a4eda15044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:09.590690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d1e22-a5f9-411f-94b9-85a4eda15044 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:09.608423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 713/1431] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:10.489226 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97c523-a4c2-4147-ba42-1c79443fd413 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:10.489758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97c523-a4c2-4147-ba42-1c79443fd413 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:10.489758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97c523-a4c2-4147-ba42-1c79443fd413 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:10.491060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:53:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:10.496694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbc39a-701d-44b8-a744-27013c772018 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:10.497100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbc39a-701d-44b8-a744-27013c772018 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:10.497239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbc39a-701d-44b8-a744-27013c772018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:10.573886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8ecbc39a-701d-44b8-a744-27013c772018 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:10.575139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8ecbc39a-701d-44b8-a744-27013c772018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:10.641701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:10.669768 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:53:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:10.822568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97c523-a4c2-4147-ba42-1c79443fd413 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:10.824125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0e97c523-a4c2-4147-ba42-1c79443fd413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:10.842317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:10.970356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97c523-a4c2-4147-ba42-1c79443fd413 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:11.057253 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:11.083495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2987a0a4-9923-4809-a0fa-d06464e0844a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:11.084035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987a0a4-9923-4809-a0fa-d06464e0844a service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:11.084309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987a0a4-9923-4809-a0fa-d06464e0844a service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:11.084575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2987a0a4-9923-4809-a0fa-d06464e0844a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:11.086177 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2987a0a4-9923-4809-a0fa-d06464e0844a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:11.174591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987a0a4-9923-4809-a0fa-d06464e0844a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:11.176222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2987a0a4-9923-4809-a0fa-d06464e0844a service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:11.195035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 720/1434] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:12.059406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6737f8-a136-40c5-8699-72cd0e6f4b4c None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:12.059957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6737f8-a136-40c5-8699-72cd0e6f4b4c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:12.059957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6737f8-a136-40c5-8699-72cd0e6f4b4c None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:12.061346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:53:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:12.066393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00992237-142a-4126-b5ac-61e3445e98cf None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:12.066774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00992237-142a-4126-b5ac-61e3445e98cf None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:12.066936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00992237-142a-4126-b5ac-61e3445e98cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:12.142430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-00992237-142a-4126-b5ac-61e3445e98cf None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:12.143695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-00992237-142a-4126-b5ac-61e3445e98cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:12.186729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:12.242160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:53:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:12.392861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6737f8-a136-40c5-8699-72cd0e6f4b4c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:12.394316 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ca6737f8-a136-40c5-8699-72cd0e6f4b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:12.538898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6737f8-a136-40c5-8699-72cd0e6f4b4c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:12.595585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:12.773913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355affe-9b11-490b-8170-20c92d6537b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:12.774643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355affe-9b11-490b-8170-20c92d6537b5 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:12.775062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355affe-9b11-490b-8170-20c92d6537b5 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:12.775342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355affe-9b11-490b-8170-20c92d6537b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:12.776912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-1355affe-9b11-490b-8170-20c92d6537b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:12.864037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355affe-9b11-490b-8170-20c92d6537b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:12.865590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355affe-9b11-490b-8170-20c92d6537b5 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:12.884622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 716/1437] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:13.768901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e4e03-95b2-48b5-b7d8-7a46defc812d None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:13.769454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e4e03-95b2-48b5-b7d8-7a46defc812d None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:13.769454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e4e03-95b2-48b5-b7d8-7a46defc812d None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:13.770771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:53:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:13.776237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8508da-be23-4cd4-8819-58e332fc81d0 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:13.776630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8508da-be23-4cd4-8819-58e332fc81d0 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:13.776867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8508da-be23-4cd4-8819-58e332fc81d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:13.853370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-cb8508da-be23-4cd4-8819-58e332fc81d0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:13.853370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-cb8508da-be23-4cd4-8819-58e332fc81d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:13.854661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:13.947447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:53:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:14.099256 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e4e03-95b2-48b5-b7d8-7a46defc812d None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:14.100825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f17e4e03-95b2-48b5-b7d8-7a46defc812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:14.247593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e4e03-95b2-48b5-b7d8-7a46defc812d None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:14.275799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:14.360584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963c81b-51c9-4d98-92a7-c7ae86eb694a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:14.361193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963c81b-51c9-4d98-92a7-c7ae86eb694a service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:14.361471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963c81b-51c9-4d98-92a7-c7ae86eb694a service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:14.361742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963c81b-51c9-4d98-92a7-c7ae86eb694a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:14.363398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a963c81b-51c9-4d98-92a7-c7ae86eb694a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:14.452860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963c81b-51c9-4d98-92a7-c7ae86eb694a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:14.454364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963c81b-51c9-4d98-92a7-c7ae86eb694a service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:14.472065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 723/1440] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:16.313788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d56efd-d12c-43cc-9afb-e9a47167ae0d None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:16.314335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d56efd-d12c-43cc-9afb-e9a47167ae0d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:16.314335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d56efd-d12c-43cc-9afb-e9a47167ae0d None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:16.315720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:16.320993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1122ccf-6183-4762-aaa8-ecbd28c81ea8 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:16.322030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1122ccf-6183-4762-aaa8-ecbd28c81ea8 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:16.322364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1122ccf-6183-4762-aaa8-ecbd28c81ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:16.362868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:16.388743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-a1122ccf-6183-4762-aaa8-ecbd28c81ea8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:16.389970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a1122ccf-6183-4762-aaa8-ecbd28c81ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:16.488533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:16.798809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d56efd-d12c-43cc-9afb-e9a47167ae0d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:16.800404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-23d56efd-d12c-43cc-9afb-e9a47167ae0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:16.902584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d56efd-d12c-43cc-9afb-e9a47167ae0d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:16.934526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:16.975756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddf48c-736a-4b74-81ef-377c02917f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:16.976351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddf48c-736a-4b74-81ef-377c02917f18 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:16.976625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddf48c-736a-4b74-81ef-377c02917f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:16.976891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddf48c-736a-4b74-81ef-377c02917f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:16.978484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b5ddf48c-736a-4b74-81ef-377c02917f18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:17.085815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddf48c-736a-4b74-81ef-377c02917f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:17.087511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ddf48c-736a-4b74-81ef-377c02917f18 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:17.105612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 719/1443] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:19.212762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be316dd-8cdf-4060-8007-49e795774cb9 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:19.213259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be316dd-8cdf-4060-8007-49e795774cb9 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:19.213366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be316dd-8cdf-4060-8007-49e795774cb9 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:19.214685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:19.220587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55c97-35fb-411c-a147-97aa458f1a45 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:19.221007 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55c97-35fb-411c-a147-97aa458f1a45 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:19.221115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a55c97-35fb-411c-a147-97aa458f1a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:19.285060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-28a55c97-35fb-411c-a147-97aa458f1a45 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:19.286270 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-28a55c97-35fb-411c-a147-97aa458f1a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:19.299205 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:19.381134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:19.665670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be316dd-8cdf-4060-8007-49e795774cb9 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:19.667243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6be316dd-8cdf-4060-8007-49e795774cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:19.769211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be316dd-8cdf-4060-8007-49e795774cb9 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:19.782425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:19.844753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a038c-c28e-425a-b4e6-4bf43661b134 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:19.845275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a038c-c28e-425a-b4e6-4bf43661b134 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:19.845547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a038c-c28e-425a-b4e6-4bf43661b134 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:19.845810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a038c-c28e-425a-b4e6-4bf43661b134 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:19.847424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-853a038c-c28e-425a-b4e6-4bf43661b134 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:19.958399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a038c-c28e-425a-b4e6-4bf43661b134 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:19.959943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a038c-c28e-425a-b4e6-4bf43661b134 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:19.978537 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 726/1446] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:22.328469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd124e8-e3f2-4a1c-8eac-539b1a9b6d54 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:22.329020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd124e8-e3f2-4a1c-8eac-539b1a9b6d54 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:22.329020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd124e8-e3f2-4a1c-8eac-539b1a9b6d54 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:22.330347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:22.335647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab9221-0f13-499e-8232-d8e308606f2b None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:22.336075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab9221-0f13-499e-8232-d8e308606f2b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:22.336197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ab9221-0f13-499e-8232-d8e308606f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:22.383213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:22.402410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-a6ab9221-0f13-499e-8232-d8e308606f2b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:22.403643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a6ab9221-0f13-499e-8232-d8e308606f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:22.499058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:22.784447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd124e8-e3f2-4a1c-8eac-539b1a9b6d54 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:22.785962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3dd124e8-e3f2-4a1c-8eac-539b1a9b6d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:22.890203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd124e8-e3f2-4a1c-8eac-539b1a9b6d54 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:22.931598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:22.965311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b37a1-18e6-4698-b98c-23c3ee2229f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:22.965913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b37a1-18e6-4698-b98c-23c3ee2229f1 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:22.966186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b37a1-18e6-4698-b98c-23c3ee2229f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:22.966448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b37a1-18e6-4698-b98c-23c3ee2229f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:22.968121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-135b37a1-18e6-4698-b98c-23c3ee2229f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:23.074793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b37a1-18e6-4698-b98c-23c3ee2229f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:23.076239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b37a1-18e6-4698-b98c-23c3ee2229f1 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:23.095184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 722/1449] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:25.117694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d30d-c0e4-400d-951d-ebcdbb74bc07 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:25.119127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d30d-c0e4-400d-951d-ebcdbb74bc07 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:25.119127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893d30d-c0e4-400d-951d-ebcdbb74bc07 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:25.120347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:25.125545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304a34d-dabb-40ea-94dd-ff20f70c4979 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:25.125970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304a34d-dabb-40ea-94dd-ff20f70c4979 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:25.126090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304a34d-dabb-40ea-94dd-ff20f70c4979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:25.191509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b304a34d-dabb-40ea-94dd-ff20f70c4979 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:25.192581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b304a34d-dabb-40ea-94dd-ff20f70c4979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:25.213756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:25.287356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:25.569528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893d30d-c0e4-400d-951d-ebcdbb74bc07 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:25.571188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8893d30d-c0e4-400d-951d-ebcdbb74bc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:25.673042 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893d30d-c0e4-400d-951d-ebcdbb74bc07 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:25.692267 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:25.745304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b337057-341f-4b57-aee9-c5ca50192648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:25.745902 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b337057-341f-4b57-aee9-c5ca50192648 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:25.746175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b337057-341f-4b57-aee9-c5ca50192648 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:25.746439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b337057-341f-4b57-aee9-c5ca50192648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:25.748054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6b337057-341f-4b57-aee9-c5ca50192648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:25.857842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b337057-341f-4b57-aee9-c5ca50192648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:25.859307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b337057-341f-4b57-aee9-c5ca50192648 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:25.877606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 729/1452] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:27.924050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5bfe5-68f3-4c93-8312-847140df8a4d None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:27.924575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5bfe5-68f3-4c93-8312-847140df8a4d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:27.924575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5bfe5-68f3-4c93-8312-847140df8a4d None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:27.925858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:27.931551 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472b9af-56f1-4fcd-b135-1c340c6317f4 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:27.931957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472b9af-56f1-4fcd-b135-1c340c6317f4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:27.932105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472b9af-56f1-4fcd-b135-1c340c6317f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:27.995406 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:27.998450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-6472b9af-56f1-4fcd-b135-1c340c6317f4 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:27.999753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6472b9af-56f1-4fcd-b135-1c340c6317f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:28.096153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:28.380941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5bfe5-68f3-4c93-8312-847140df8a4d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:28.382462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ebe5bfe5-68f3-4c93-8312-847140df8a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:28.486295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5bfe5-68f3-4c93-8312-847140df8a4d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:28.535100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:28.561456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd9c04-ffc5-4dcb-9489-a7cb7c579b42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:28.562073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd9c04-ffc5-4dcb-9489-a7cb7c579b42 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:28.562370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd9c04-ffc5-4dcb-9489-a7cb7c579b42 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:28.562640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd9c04-ffc5-4dcb-9489-a7cb7c579b42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:28.564270 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-09cd9c04-ffc5-4dcb-9489-a7cb7c579b42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:28.673619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd9c04-ffc5-4dcb-9489-a7cb7c579b42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:28.675062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd9c04-ffc5-4dcb-9489-a7cb7c579b42 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:28.692749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 725/1455] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:30.667906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e238-9f7e-4ca4-90ce-138d4b915935 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:30.668444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e238-9f7e-4ca4-90ce-138d4b915935 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:30.668444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e238-9f7e-4ca4-90ce-138d4b915935 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:30.669793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:30.675417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579cf45-aaeb-4a02-926c-19a5e4116540 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:30.675866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579cf45-aaeb-4a02-926c-19a5e4116540 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:30.675976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579cf45-aaeb-4a02-926c-19a5e4116540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:30.740582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-a579cf45-aaeb-4a02-926c-19a5e4116540 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:30.741777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a579cf45-aaeb-4a02-926c-19a5e4116540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:30.770475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:30.837030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:31.125657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28e238-9f7e-4ca4-90ce-138d4b915935 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:31.127224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5a28e238-9f7e-4ca4-90ce-138d4b915935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:31.230707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28e238-9f7e-4ca4-90ce-138d4b915935 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:31.258624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:31.303517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed0ea3-57ec-4a8a-b74e-35a2afac08b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:31.304118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed0ea3-57ec-4a8a-b74e-35a2afac08b0 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:31.304398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed0ea3-57ec-4a8a-b74e-35a2afac08b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:31.304667 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed0ea3-57ec-4a8a-b74e-35a2afac08b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:31.306823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-19ed0ea3-57ec-4a8a-b74e-35a2afac08b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:31.415591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed0ea3-57ec-4a8a-b74e-35a2afac08b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:31.416621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed0ea3-57ec-4a8a-b74e-35a2afac08b0 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:31.435098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 732/1458] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:33.402462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49854a-7500-4ed8-8093-bb406d964d94 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:33.402997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49854a-7500-4ed8-8093-bb406d964d94 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:33.402997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49854a-7500-4ed8-8093-bb406d964d94 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:33.404424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:33.409852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895190f-2e88-4302-9f4d-0eb2f7f231b3 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:33.410271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895190f-2e88-4302-9f4d-0eb2f7f231b3 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:33.410417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895190f-2e88-4302-9f4d-0eb2f7f231b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:33.475704 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5895190f-2e88-4302-9f4d-0eb2f7f231b3 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:33.476836 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5895190f-2e88-4302-9f4d-0eb2f7f231b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:33.485409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:33.575418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:33.856874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49854a-7500-4ed8-8093-bb406d964d94 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:33.858465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9f49854a-7500-4ed8-8093-bb406d964d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:33.961180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49854a-7500-4ed8-8093-bb406d964d94 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:34.015910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:34.033651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ad8bc-44f1-471c-8e7e-c888a87d5cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:34.034253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ad8bc-44f1-471c-8e7e-c888a87d5cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:34.034525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ad8bc-44f1-471c-8e7e-c888a87d5cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:34.034790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ad8bc-44f1-471c-8e7e-c888a87d5cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:34.036392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-544ad8bc-44f1-471c-8e7e-c888a87d5cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:34.143369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ad8bc-44f1-471c-8e7e-c888a87d5cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:34.144721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ad8bc-44f1-471c-8e7e-c888a87d5cf8 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:34.162617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 728/1461] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:36.143133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98e57-86e6-416d-aad1-89251c6905bf None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:36.143732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98e57-86e6-416d-aad1-89251c6905bf None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:36.143732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98e57-86e6-416d-aad1-89251c6905bf None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:36.145017 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:36.150926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e5a99-c2ce-4aa4-96eb-bd230c987dd5 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:36.151400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e5a99-c2ce-4aa4-96eb-bd230c987dd5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:36.151505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e5a99-c2ce-4aa4-96eb-bd230c987dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:36.219843 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7f7e5a99-c2ce-4aa4-96eb-bd230c987dd5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:36.221055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7f7e5a99-c2ce-4aa4-96eb-bd230c987dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:36.259260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:36.316629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:36.619418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d98e57-86e6-416d-aad1-89251c6905bf None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:36.621011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-59d98e57-86e6-416d-aad1-89251c6905bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:36.727826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d98e57-86e6-416d-aad1-89251c6905bf None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:36.768539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:36.808079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd6ca9-dfc9-455a-869e-a5343110537f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:36.808682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd6ca9-dfc9-455a-869e-a5343110537f service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:36.808954 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd6ca9-dfc9-455a-869e-a5343110537f service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:36.809218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd6ca9-dfc9-455a-869e-a5343110537f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:36.810822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-92fd6ca9-dfc9-455a-869e-a5343110537f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:36.927605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd6ca9-dfc9-455a-869e-a5343110537f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:36.928939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd6ca9-dfc9-455a-869e-a5343110537f service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:36.949900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 735/1464] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:38.929619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994f19c-11d7-401c-a4a4-682fececdbf1 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:38.930665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994f19c-11d7-401c-a4a4-682fececdbf1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:38.930665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994f19c-11d7-401c-a4a4-682fececdbf1 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:38.932085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:38.937365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea333e-3304-46c0-b216-963730fd3678 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:38.938453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea333e-3304-46c0-b216-963730fd3678 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:38.938453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea333e-3304-46c0-b216-963730fd3678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:39.002392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-13ea333e-3304-46c0-b216-963730fd3678 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:39.003638 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-13ea333e-3304-46c0-b216-963730fd3678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:39.021772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:39.099419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:39.382542 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994f19c-11d7-401c-a4a4-682fececdbf1 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:39.384764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7994f19c-11d7-401c-a4a4-682fececdbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:39.488446 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994f19c-11d7-401c-a4a4-682fececdbf1 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:39.553967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:39.561213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6587d5-4e41-4cbf-8719-f3f26b70c4b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:39.568010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6587d5-4e41-4cbf-8719-f3f26b70c4b7 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:39.568010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6587d5-4e41-4cbf-8719-f3f26b70c4b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:39.568010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6587d5-4e41-4cbf-8719-f3f26b70c4b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:39.568010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ea6587d5-4e41-4cbf-8719-f3f26b70c4b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:39.669832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6587d5-4e41-4cbf-8719-f3f26b70c4b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:39.671272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6587d5-4e41-4cbf-8719-f3f26b70c4b7 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:39.689427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 731/1467] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:39 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:41.605780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50f543-f6f8-4d21-9a0a-b4ea7a4005df None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:41.606297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50f543-f6f8-4d21-9a0a-b4ea7a4005df None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:41.606411 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50f543-f6f8-4d21-9a0a-b4ea7a4005df None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:41.607707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:53:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:53:41.613073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cd1ab-339a-4e7a-851f-e2ee5ebcfa12 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:41.613506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cd1ab-339a-4e7a-851f-e2ee5ebcfa12 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:41.613612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cd1ab-339a-4e7a-851f-e2ee5ebcfa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:41.678966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-520cd1ab-339a-4e7a-851f-e2ee5ebcfa12 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:53:41.680239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-520cd1ab-339a-4e7a-851f-e2ee5ebcfa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:41.730820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:41.777825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:53:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:53:42.062718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50f543-f6f8-4d21-9a0a-b4ea7a4005df None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:53:42.064352 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cd50f543-f6f8-4d21-9a0a-b4ea7a4005df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:42.167578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50f543-f6f8-4d21-9a0a-b4ea7a4005df None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:53:42.215642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:42.241009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb912b77-1f50-4054-9b77-7585f418d1d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:53:42.241658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb912b77-1f50-4054-9b77-7585f418d1d4 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:53:42.241947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb912b77-1f50-4054-9b77-7585f418d1d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:53:42.242419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb912b77-1f50-4054-9b77-7585f418d1d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:53:42.244561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cb912b77-1f50-4054-9b77-7585f418d1d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:53:42.353345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb912b77-1f50-4054-9b77-7585f418d1d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:53:42.354770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb912b77-1f50-4054-9b77-7585f418d1d4 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:53:42.374872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 738/1470] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:53:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:53:51.785931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=22 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:53:52.372427 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=66 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:02.021800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f200d9c-2d0f-46fc-a631-a538b39d3b5e None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:02.022328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f200d9c-2d0f-46fc-a631-a538b39d3b5e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:02.022328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f200d9c-2d0f-46fc-a631-a538b39d3b5e None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:02.023674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 733/1471] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:54:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:54:02.029465 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510baf42-2c79-4fba-9060-e72dc741de60 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:02.029910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510baf42-2c79-4fba-9060-e72dc741de60 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:02.030038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510baf42-2c79-4fba-9060-e72dc741de60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:02.096512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-510baf42-2c79-4fba-9060-e72dc741de60 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:54:02.097725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-510baf42-2c79-4fba-9060-e72dc741de60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:02.194168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 739/1472] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:54:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:54:02.492632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df945d20-c1c9-4e96-a122-1250d8b59838 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:02.502008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df945d20-c1c9-4e96-a122-1250d8b59838 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:02.502008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df945d20-c1c9-4e96-a122-1250d8b59838 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:02.502008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 734/1473] 10.222.0.105 () {54 vars in 972 bytes} [Wed Aug 24 04:54:02 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:54:02.502303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b57b4-fc30-463a-9b11-e202a36625f4 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:02.502303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b57b4-fc30-463a-9b11-e202a36625f4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:02.502303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b57b4-fc30-463a-9b11-e202a36625f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:02.576385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-6b9b57b4-fc30-463a-9b11-e202a36625f4 None None] MFA Rules not processed for user `f1d62a4c07c4402ea5803f6e68fb1d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:54:02.577527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6b9b57b4-fc30-463a-9b11-e202a36625f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:02.598789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:02.655872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 740/1474] 10.222.0.105 () {58 vars in 1069 bytes} [Wed Aug 24 04:54:02 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:54:02.663540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df945d20-c1c9-4e96-a122-1250d8b59838 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:54:02.665583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-df945d20-c1c9-4e96-a122-1250d8b59838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:02.767733 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df945d20-c1c9-4e96-a122-1250d8b59838 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:54:02.840823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d9294-9279-4ec5-8f09-116fb9a1027a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d62a4c07c4402ea5803f6e68fb1d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:54:02.841353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d9294-9279-4ec5-8f09-116fb9a1027a service glance] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:02.841629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d9294-9279-4ec5-8f09-116fb9a1027a service glance] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:02.841898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d9294-9279-4ec5-8f09-116fb9a1027a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:02.843521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-621d9294-9279-4ec5-8f09-116fb9a1027a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:02.895915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:02.950572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d9294-9279-4ec5-8f09-116fb9a1027a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:54:02.952025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d9294-9279-4ec5-8f09-116fb9a1027a service glance] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:54:02.970485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 735/1475] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:54:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:54:04.978117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a77e8a-ba70-43dd-b62b-b16fdbba8751 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:04.978709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a77e8a-ba70-43dd-b62b-b16fdbba8751 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:04.978709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a77e8a-ba70-43dd-b62b-b16fdbba8751 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:04.980096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 741/1476] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:54:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:54:04.985562 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34ea2-90f3-4382-ac8d-a3cd86bbd70e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:04.985994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34ea2-90f3-4382-ac8d-a3cd86bbd70e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:04.986168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf34ea2-90f3-4382-ac8d-a3cd86bbd70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:05.056500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-0cf34ea2-90f3-4382-ac8d-a3cd86bbd70e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:54:05.057717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0cf34ea2-90f3-4382-ac8d-a3cd86bbd70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:05.152976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 736/1477] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:54:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:54:05.425326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a77e8a-ba70-43dd-b62b-b16fdbba8751 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:54:05.426797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d3a77e8a-ba70-43dd-b62b-b16fdbba8751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:05.549839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a77e8a-ba70-43dd-b62b-b16fdbba8751 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:54:05.600402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:05.644104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802cda1-9485-43f9-aa86-386834c3a564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:54:05.644712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802cda1-9485-43f9-aa86-386834c3a564 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:05.644989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802cda1-9485-43f9-aa86-386834c3a564 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:05.645258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802cda1-9485-43f9-aa86-386834c3a564 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:05.645889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802cda1-9485-43f9-aa86-386834c3a564 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:54:05.647649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WARNING py.warnings [None req-2802cda1-9485-43f9-aa86-386834c3a564 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:54:05.647649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: warnings.warn(msg) Aug 24 04:54:05.647649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: Aug 24 04:54:05.648780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802cda1-9485-43f9-aa86-386834c3a564 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:54:05.668756 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 742/1478] 10.222.0.105 () {56 vars in 1113 bytes} [Wed Aug 24 04:54:05 2022] GET /identity/v3/projects => generated 2562 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:54:07.606897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0a895-d3d7-4d86-ba49-37da197e362f None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:07.607428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0a895-d3d7-4d86-ba49-37da197e362f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:07.607541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb0a895-d3d7-4d86-ba49-37da197e362f None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:07.608864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 737/1479] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:54:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:54:07.614223 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f316-876f-461f-a920-8c84c588ee68 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:07.614628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f316-876f-461f-a920-8c84c588ee68 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:07.614774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27f316-876f-461f-a920-8c84c588ee68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:07.679795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5e27f316-876f-461f-a920-8c84c588ee68 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:54:07.681038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5e27f316-876f-461f-a920-8c84c588ee68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:07.776117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 743/1480] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:54:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:54:08.070166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0a895-d3d7-4d86-ba49-37da197e362f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:54:08.071766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2fb0a895-d3d7-4d86-ba49-37da197e362f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:08.081317 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:08.218915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb0a895-d3d7-4d86-ba49-37da197e362f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:54:08.300342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:08.332622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e3c14-afd4-4849-a18a-c918be164919 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:54:08.333253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e3c14-afd4-4849-a18a-c918be164919 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:08.333528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e3c14-afd4-4849-a18a-c918be164919 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:08.333816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e3c14-afd4-4849-a18a-c918be164919 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:08.335458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a88e3c14-afd4-4849-a18a-c918be164919 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:08.425057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e3c14-afd4-4849-a18a-c918be164919 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:54:08.426617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88e3c14-afd4-4849-a18a-c918be164919 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:54:08.445061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 738/1481] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:54:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:54:10.549146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd98fc-dc35-407d-aa2d-241b173215e5 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:10.549146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd98fc-dc35-407d-aa2d-241b173215e5 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:10.549146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd98fc-dc35-407d-aa2d-241b173215e5 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:10.549146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 744/1482] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:54:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:54:10.550969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177185f-5364-468a-8f91-3ab80c9c39b8 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:10.551387 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177185f-5364-468a-8f91-3ab80c9c39b8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:10.551536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177185f-5364-468a-8f91-3ab80c9c39b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:10.616941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6177185f-5364-468a-8f91-3ab80c9c39b8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:54:10.618297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6177185f-5364-468a-8f91-3ab80c9c39b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:10.664029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:10.715041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 739/1483] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:54:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:54:11.000073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd98fc-dc35-407d-aa2d-241b173215e5 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:54:11.001681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a7fd98fc-dc35-407d-aa2d-241b173215e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:11.026206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:11.106062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd98fc-dc35-407d-aa2d-241b173215e5 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:54:11.179890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5792b05e-0b69-498d-bf3c-8a8e09d4518f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:54:11.181077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b05e-0b69-498d-bf3c-8a8e09d4518f service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:11.181348 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b05e-0b69-498d-bf3c-8a8e09d4518f service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:11.181797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5792b05e-0b69-498d-bf3c-8a8e09d4518f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:11.183427 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5792b05e-0b69-498d-bf3c-8a8e09d4518f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:11.258313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:11.292949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792b05e-0b69-498d-bf3c-8a8e09d4518f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:54:11.294655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5792b05e-0b69-498d-bf3c-8a8e09d4518f service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:54:11.313904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 745/1484] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:54:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:54:13.268884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dee68-a3f5-47cd-9f4a-14ac6add0024 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:13.269416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dee68-a3f5-47cd-9f4a-14ac6add0024 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:13.269416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dee68-a3f5-47cd-9f4a-14ac6add0024 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:13.270718 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 740/1485] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:54:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:54:13.275955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69108df-8d37-4289-af7e-0f8642cc3aca None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:13.276364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69108df-8d37-4289-af7e-0f8642cc3aca None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:13.276501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69108df-8d37-4289-af7e-0f8642cc3aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:13.342161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d69108df-8d37-4289-af7e-0f8642cc3aca None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:54:13.343392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d69108df-8d37-4289-af7e-0f8642cc3aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:13.440506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 746/1486] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:54:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:54:13.728668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dee68-a3f5-47cd-9f4a-14ac6add0024 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:54:13.730321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4c7dee68-a3f5-47cd-9f4a-14ac6add0024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:13.833926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dee68-a3f5-47cd-9f4a-14ac6add0024 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:54:13.907543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:13.909305 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244894a-f5cf-45b7-a4f5-2760c04541d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:54:13.909907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244894a-f5cf-45b7-a4f5-2760c04541d6 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:13.910182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244894a-f5cf-45b7-a4f5-2760c04541d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:13.910448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244894a-f5cf-45b7-a4f5-2760c04541d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:13.912112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b244894a-f5cf-45b7-a4f5-2760c04541d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:14.022037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244894a-f5cf-45b7-a4f5-2760c04541d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:54:14.023492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244894a-f5cf-45b7-a4f5-2760c04541d6 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:54:14.041929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 741/1487] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:54:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:54:16.043897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563b33d-f4a5-4e8e-a1c0-257b28454bbe None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:16.044448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563b33d-f4a5-4e8e-a1c0-257b28454bbe None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:16.044448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563b33d-f4a5-4e8e-a1c0-257b28454bbe None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:16.045745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 747/1488] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:54:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:54:16.051448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cacd3-8e2f-4d06-bb1a-8e82b05255b8 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:16.051869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cacd3-8e2f-4d06-bb1a-8e82b05255b8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:16.051974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cacd3-8e2f-4d06-bb1a-8e82b05255b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:16.116660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6f0cacd3-8e2f-4d06-bb1a-8e82b05255b8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:54:16.118478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6f0cacd3-8e2f-4d06-bb1a-8e82b05255b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:16.169857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:16.215752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 742/1489] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:54:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:54:16.506797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563b33d-f4a5-4e8e-a1c0-257b28454bbe None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:54:16.508367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e563b33d-f4a5-4e8e-a1c0-257b28454bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:16.545491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:16.614488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e563b33d-f4a5-4e8e-a1c0-257b28454bbe None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:54:16.686718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927de76-9fa9-4533-a53b-a99f5b270c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:54:16.687419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927de76-9fa9-4533-a53b-a99f5b270c7c service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:16.687751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927de76-9fa9-4533-a53b-a99f5b270c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:16.688026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927de76-9fa9-4533-a53b-a99f5b270c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:16.689790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6927de76-9fa9-4533-a53b-a99f5b270c7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:16.770727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:16.801319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927de76-9fa9-4533-a53b-a99f5b270c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:54:16.802736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6927de76-9fa9-4533-a53b-a99f5b270c7c service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:54:16.820792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 748/1490] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:54:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:54:19.220523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64352c-42f4-47d9-bf9b-6c4b254f75a7 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:19.221014 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64352c-42f4-47d9-bf9b-6c4b254f75a7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:19.221113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64352c-42f4-47d9-bf9b-6c4b254f75a7 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:19.222403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 743/1491] 10.222.0.105 () {54 vars in 932 bytes} [Wed Aug 24 04:54:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:54:19.227681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09354e7d-0387-42cf-b685-f2446b226d90 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:19.228131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09354e7d-0387-42cf-b685-f2446b226d90 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:19.228239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09354e7d-0387-42cf-b685-f2446b226d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:19.296584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-09354e7d-0387-42cf-b685-f2446b226d90 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:54:19.297829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-09354e7d-0387-42cf-b685-f2446b226d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:19.400591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 749/1492] 10.222.0.105 () {58 vars in 1029 bytes} [Wed Aug 24 04:54:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:54:19.704008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64352c-42f4-47d9-bf9b-6c4b254f75a7 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:54:19.705570 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-bc64352c-42f4-47d9-bf9b-6c4b254f75a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:19.807923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64352c-42f4-47d9-bf9b-6c4b254f75a7 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:54:19.880870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef7501-2b69-4da2-b896-daba3c371b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:54:19.881447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef7501-2b69-4da2-b896-daba3c371b83 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:54:19.881729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef7501-2b69-4da2-b896-daba3c371b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:54:19.882004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef7501-2b69-4da2-b896-daba3c371b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:54:19.883662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-daef7501-2b69-4da2-b896-daba3c371b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:54:19.893316 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:19.992239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef7501-2b69-4da2-b896-daba3c371b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:54:19.993653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef7501-2b69-4da2-b896-daba3c371b83 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:54:20.011895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 744/1493] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:54:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:54:29.397667 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=82 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:54:30.009682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=50 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:44.229737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf613371-532e-4f70-8b9e-23d936fc9f53 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:44.230262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf613371-532e-4f70-8b9e-23d936fc9f53 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:44.230262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf613371-532e-4f70-8b9e-23d936fc9f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:44.306340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-cf613371-532e-4f70-8b9e-23d936fc9f53 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:55:44.307569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cf613371-532e-4f70-8b9e-23d936fc9f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:44.402834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 750/1494] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:55:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:55:44.411881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc28a0-8e79-4f43-8371-1103af282fdd None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:44.412370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc28a0-8e79-4f43-8371-1103af282fdd None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:44.412474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc28a0-8e79-4f43-8371-1103af282fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:44.487006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-53dc28a0-8e79-4f43-8371-1103af282fdd None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:55:44.488163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-53dc28a0-8e79-4f43-8371-1103af282fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:44.582063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 745/1495] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:55:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:55:44.586642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf613371-532e-4f70-8b9e-23d936fc9f53 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:55:44.587875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cf613371-532e-4f70-8b9e-23d936fc9f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:44.600118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf613371-532e-4f70-8b9e-23d936fc9f53 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:55:44.661147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:44.694389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:55:44.694998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:44.695289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:44.695556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:44.696196 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:55:44.696492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:55:44.697494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WARNING py.warnings [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:55:44.697494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: warnings.warn(msg) Aug 24 04:55:44.697494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: Aug 24 04:55:44.698368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:55:44.709895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 751/1496] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:55:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:55:44.719057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc28a0-8e79-4f43-8371-1103af282fdd None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:55:44.727919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc28a0-8e79-4f43-8371-1103af282fdd None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:55:44.788977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:44.823469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a3de8-f247-4019-83e6-848b6eb49ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:55:44.824067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a3de8-f247-4019-83e6-848b6eb49ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:44.824340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a3de8-f247-4019-83e6-848b6eb49ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:44.824605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a3de8-f247-4019-83e6-848b6eb49ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:44.825217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a3de8-f247-4019-83e6-848b6eb49ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:55:44.827555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: WARNING py.warnings [None req-9f5a3de8-f247-4019-83e6-848b6eb49ca4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:55:44.827555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: warnings.warn(msg) Aug 24 04:55:44.827555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: Aug 24 04:55:44.828740 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a3de8-f247-4019-83e6-848b6eb49ca4 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:55:44.959174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 746/1497] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:55:44 2022] POST /identity/v3/projects => generated 380 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:55:44.963442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:55:44.964949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:44.997806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad802c7f-34d4-41e9-8690-d431d7dba2a3 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:55:45.092451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce05c1-2f9a-4dfc-b9be-c8ec027d07dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:55:45.104096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce05c1-2f9a-4dfc-b9be-c8ec027d07dd admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:45.104096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce05c1-2f9a-4dfc-b9be-c8ec027d07dd admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:45.104096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce05c1-2f9a-4dfc-b9be-c8ec027d07dd admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:45.104096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce05c1-2f9a-4dfc-b9be-c8ec027d07dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:55:45.104096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WARNING py.warnings [None req-99ce05c1-2f9a-4dfc-b9be-c8ec027d07dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:55:45.104096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: warnings.warn(msg) Aug 24 04:55:45.104096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: Aug 24 04:55:45.104751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce05c1-2f9a-4dfc-b9be-c8ec027d07dd admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:55:45.289160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 752/1498] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:55:44 2022] POST /identity/v3/users => generated 291 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:55:45.293387 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a3de8-f247-4019-83e6-848b6eb49ca4 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:55:45.294894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9f5a3de8-f247-4019-83e6-848b6eb49ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:45.358868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a3de8-f247-4019-83e6-848b6eb49ca4 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:55:45.442899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:45.452956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7b5c3-3a2d-4159-8a83-475b95bb0f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:55:45.457103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b5c3-3a2d-4159-8a83-475b95bb0f6e admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:45.457103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b5c3-3a2d-4159-8a83-475b95bb0f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:45.457103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7b5c3-3a2d-4159-8a83-475b95bb0f6e admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:45.457103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7b5c3-3a2d-4159-8a83-475b95bb0f6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:55:45.457103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: WARNING py.warnings [None req-aed7b5c3-3a2d-4159-8a83-475b95bb0f6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:55:45.457103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: warnings.warn(msg) Aug 24 04:55:45.457103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: Aug 24 04:55:45.457714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed7b5c3-3a2d-4159-8a83-475b95bb0f6e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:55:45.469235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 747/1499] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:55:45 2022] GET /identity/v3/roles => generated 1427 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:55:45.473223 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce05c1-2f9a-4dfc-b9be-c8ec027d07dd admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:55:45.474727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-99ce05c1-2f9a-4dfc-b9be-c8ec027d07dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:45.486718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce05c1-2f9a-4dfc-b9be-c8ec027d07dd admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:55:45.511066 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:45.581734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a756133d-9104-451e-b215-a589f3b52057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:55:45.582347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756133d-9104-451e-b215-a589f3b52057 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:45.582621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756133d-9104-451e-b215-a589f3b52057 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:45.582890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756133d-9104-451e-b215-a589f3b52057 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:45.583558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756133d-9104-451e-b215-a589f3b52057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:55:45.584562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WARNING py.warnings [None req-a756133d-9104-451e-b215-a589f3b52057 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:55:45.584562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: warnings.warn(msg) Aug 24 04:55:45.584562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: Aug 24 04:55:45.585450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a756133d-9104-451e-b215-a589f3b52057 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:55:45.599115 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 753/1500] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:55:45 2022] GET /identity/v3/roles => generated 1427 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:55:45.603265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7b5c3-3a2d-4159-8a83-475b95bb0f6e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:55:45.604783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-aed7b5c3-3a2d-4159-8a83-475b95bb0f6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:45.638407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7b5c3-3a2d-4159-8a83-475b95bb0f6e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:55:45.731234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1562f8-4c17-4f2c-bf9b-41f057ed4290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:55:45.731887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1562f8-4c17-4f2c-bf9b-41f057ed4290 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:45.732163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1562f8-4c17-4f2c-bf9b-41f057ed4290 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:45.732430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1562f8-4c17-4f2c-bf9b-41f057ed4290 admin admin] PATH_INFO: `/v3/projects/13bffd823c354fee84f689cd2006746d/users/d4fd6f2eac9d4340a36f251e75532314/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:45.754949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1562f8-4c17-4f2c-bf9b-41f057ed4290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13bffd823c354fee84f689cd2006746d, user_id=d4fd6f2eac9d4340a36f251e75532314, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:55:45.755988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: WARNING py.warnings [None req-1b1562f8-4c17-4f2c-bf9b-41f057ed4290 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:55:45.755988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: warnings.warn(msg) Aug 24 04:55:45.755988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: Aug 24 04:55:45.757921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1562f8-4c17-4f2c-bf9b-41f057ed4290 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:55:45.865304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 748/1501] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:55:45 2022] PUT /identity/v3/projects/13bffd823c354fee84f689cd2006746d/users/d4fd6f2eac9d4340a36f251e75532314/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:55:45.877926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d473a2-95f9-4923-8880-ab6c31b31a83 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:45.880808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d473a2-95f9-4923-8880-ab6c31b31a83 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:45.880808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d473a2-95f9-4923-8880-ab6c31b31a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:45.935071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-55d473a2-95f9-4923-8880-ab6c31b31a83 None None] MFA Rules not processed for user `d4fd6f2eac9d4340a36f251e75532314`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:55:45.937328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-55d473a2-95f9-4923-8880-ab6c31b31a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:46.014968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 754/1502] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:55:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:55:46.027814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042af8d-7d45-4f6b-be67-c57cc8dfc5f6 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:46.028099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042af8d-7d45-4f6b-be67-c57cc8dfc5f6 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:46.028371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042af8d-7d45-4f6b-be67-c57cc8dfc5f6 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:46.029701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 749/1503] 10.222.0.105 () {54 vars in 969 bytes} [Wed Aug 24 04:55:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:55:46.034647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed19320-4856-43f9-be3b-8d28bdfb4989 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:46.034938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed19320-4856-43f9-be3b-8d28bdfb4989 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:46.035260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed19320-4856-43f9-be3b-8d28bdfb4989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:46.067803 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:46.147924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5ed19320-4856-43f9-be3b-8d28bdfb4989 None None] MFA Rules not processed for user `29af87e1e05b4880b9ec96865994b635`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:55:46.149156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5ed19320-4856-43f9-be3b-8d28bdfb4989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:46.225917 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 755/1504] 10.222.0.105 () {58 vars in 1066 bytes} [Wed Aug 24 04:55:46 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:55:46.234645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042af8d-7d45-4f6b-be67-c57cc8dfc5f6 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:55:46.236189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6042af8d-7d45-4f6b-be67-c57cc8dfc5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:46.285423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:46.336995 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042af8d-7d45-4f6b-be67-c57cc8dfc5f6 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:55:46.410891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8518d-58a4-4d73-b74d-a9dcf284614f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:55:46.411528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8518d-58a4-4d73-b74d-a9dcf284614f service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:46.411804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8518d-58a4-4d73-b74d-a9dcf284614f service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:46.412070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8518d-58a4-4d73-b74d-a9dcf284614f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:46.413654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-40e8518d-58a4-4d73-b74d-a9dcf284614f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:46.512018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8518d-58a4-4d73-b74d-a9dcf284614f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:55:46.513356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8518d-58a4-4d73-b74d-a9dcf284614f service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:55:46.517383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:46.531733 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 750/1505] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:55:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:55:46.577289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f1a7-b9a7-4f92-b3db-2750af66a147 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:46.577705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f1a7-b9a7-4f92-b3db-2750af66a147 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:46.577838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3f1a7-b9a7-4f92-b3db-2750af66a147 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:46.579111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 756/1506] 10.222.0.105 () {52 vars in 835 bytes} [Wed Aug 24 04:55:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:55:46.582609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8518d-58a4-4d73-b74d-a9dcf284614f service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:55:46.583866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-40e8518d-58a4-4d73-b74d-a9dcf284614f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:46.681514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8518d-58a4-4d73-b74d-a9dcf284614f service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:55:46.773576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:55:46.774149 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:46.774430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:46.774702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] PATH_INFO: `/v3/users/d4fd6f2eac9d4340a36f251e75532314` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:46.787514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4fd6f2eac9d4340a36f251e75532314)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:55:46.788520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: WARNING py.warnings [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:55:46.788520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: warnings.warn(msg) Aug 24 04:55:46.788520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: Aug 24 04:55:46.789732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:55:46.799793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:47.087970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4fd6f2eac9d4340a36f251e75532314'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:55:47.088415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4fd6f2eac9d4340a36f251e75532314'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:55:47.088945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4fd6f2eac9d4340a36f251e75532314'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:55:47.207743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4fd6f2eac9d4340a36f251e75532314'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:55:47.230535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4fd6f2eac9d4340a36f251e75532314'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:55:47.250481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 751/1507] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 04:55:46 2022] DELETE /identity/v3/users/d4fd6f2eac9d4340a36f251e75532314 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:55:47.255827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe4889-6395-4945-83c5-7287ef057c2f None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:47.255827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe4889-6395-4945-83c5-7287ef057c2f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:47.255827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe4889-6395-4945-83c5-7287ef057c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:47.312873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d7fe4889-6395-4945-83c5-7287ef057c2f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:55:47.314076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d7fe4889-6395-4945-83c5-7287ef057c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:47.391252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:47.403770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 757/1508] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:55:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:55:47.413574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:55:47.414785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:47.517648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a23c94-d3ad-4e17-8b41-ff53aa81477a admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:55:47.571456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:47.590206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29fd81-298b-4cf5-9091-0d6eea4e5891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:55:47.590804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29fd81-298b-4cf5-9091-0d6eea4e5891 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:47.591107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29fd81-298b-4cf5-9091-0d6eea4e5891 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:47.591373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29fd81-298b-4cf5-9091-0d6eea4e5891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:47.593531 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6c29fd81-298b-4cf5-9091-0d6eea4e5891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:47.703568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29fd81-298b-4cf5-9091-0d6eea4e5891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:55:47.704929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29fd81-298b-4cf5-9091-0d6eea4e5891 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:55:47.723505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 752/1509] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:55:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:55:48.307578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe4889-6395-4945-83c5-7287ef057c2f None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:55:48.308956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d7fe4889-6395-4945-83c5-7287ef057c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:55:48.321709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe4889-6395-4945-83c5-7287ef057c2f None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:55:48.419942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:55:48.420679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:55:48.421208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:55:48.421500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] PATH_INFO: `/v3/projects/13bffd823c354fee84f689cd2006746d` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:55:48.431777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=13bffd823c354fee84f689cd2006746d)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:55:48.432822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WARNING py.warnings [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:55:48.432822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: warnings.warn(msg) Aug 24 04:55:48.432822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: Aug 24 04:55:48.434225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:55:48.636683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13bffd823c354fee84f689cd2006746d', 'request_id': 'req-0309c767-f4f1-4b13-8ac1-687be19b85cd'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:55:48.651957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13bffd823c354fee84f689cd2006746d', 'request_id': 'req-0309c767-f4f1-4b13-8ac1-687be19b85cd'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:55:48.658884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13bffd823c354fee84f689cd2006746d', 'request_id': 'req-0309c767-f4f1-4b13-8ac1-687be19b85cd'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:55:48.762601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] The token cache is being invalidate because project 13bffd823c354fee84f689cd2006746d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:55:48.762889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:55:48.763448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0309c767-f4f1-4b13-8ac1-687be19b85cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:55:48.765810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 758/1510] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 04:55:48 2022] DELETE /identity/v3/projects/13bffd823c354fee84f689cd2006746d => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:55:57.721168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=62 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:55:58.760566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:57:23.132198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cbe22-236a-4b1c-9188-6ec1e2d09d48 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:23.132832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cbe22-236a-4b1c-9188-6ec1e2d09d48 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:23.132832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cbe22-236a-4b1c-9188-6ec1e2d09d48 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:23.134068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:57:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:57:23.139181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4a8e6-2296-4ba1-ba90-2dcc2cf8a484 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:23.139594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4a8e6-2296-4ba1-ba90-2dcc2cf8a484 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:23.139722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb4a8e6-2296-4ba1-ba90-2dcc2cf8a484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:23.221942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-4fb4a8e6-2296-4ba1-ba90-2dcc2cf8a484 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:57:23.223190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4fb4a8e6-2296-4ba1-ba90-2dcc2cf8a484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:23.328294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:57:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:57:23.483900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cbe22-236a-4b1c-9188-6ec1e2d09d48 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:57:23.486185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-dd8cbe22-236a-4b1c-9188-6ec1e2d09d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:23.602995 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cbe22-236a-4b1c-9188-6ec1e2d09d48 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:57:23.608474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6ea5f-2a8f-4780-9ee3-e6d7dc994edc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:57:23.609074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6ea5f-2a8f-4780-9ee3-e6d7dc994edc service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:23.609353 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6ea5f-2a8f-4780-9ee3-e6d7dc994edc service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:23.609626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6ea5f-2a8f-4780-9ee3-e6d7dc994edc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:23.618373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6ea5f-2a8f-4780-9ee3-e6d7dc994edc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:57:23.619969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6ea5f-2a8f-4780-9ee3-e6d7dc994edc service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:57:23.629159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 754/1513] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:57:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:57:24.467722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fc5f6-0e74-485f-b8d7-f9eebad139b8 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:24.468244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fc5f6-0e74-485f-b8d7-f9eebad139b8 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:24.468244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fc5f6-0e74-485f-b8d7-f9eebad139b8 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:24.469542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:57:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:57:24.474645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41f9a-acf2-4b8c-a9d3-a4b31381b2c2 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:24.475058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41f9a-acf2-4b8c-a9d3-a4b31381b2c2 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:24.475213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41f9a-acf2-4b8c-a9d3-a4b31381b2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:24.494587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-22b41f9a-acf2-4b8c-a9d3-a4b31381b2c2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:57:24.495793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-22b41f9a-acf2-4b8c-a9d3-a4b31381b2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:24.503519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:57:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:57:24.655288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fc5f6-0e74-485f-b8d7-f9eebad139b8 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:57:24.669661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fc5f6-0e74-485f-b8d7-f9eebad139b8 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:57:24.675297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f87a6-1aa6-484c-8d66-294ec4acb677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:57:24.675891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f87a6-1aa6-484c-8d66-294ec4acb677 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:24.676731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f87a6-1aa6-484c-8d66-294ec4acb677 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:24.677005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f87a6-1aa6-484c-8d66-294ec4acb677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:24.685425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f87a6-1aa6-484c-8d66-294ec4acb677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:57:24.687002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f87a6-1aa6-484c-8d66-294ec4acb677 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:57:24.697125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 761/1516] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:57:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:57:25.509798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6d716-a0c0-4254-8ee6-2008d7f70e34 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:25.510373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6d716-a0c0-4254-8ee6-2008d7f70e34 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:25.510373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6d716-a0c0-4254-8ee6-2008d7f70e34 None None] PATH_INFO: `/v3` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:25.511646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 756/1517] 10.222.0.102 () {54 vars in 930 bytes} [Wed Aug 24 04:57:25 2022] GET /identity/v3 => generated 256 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:57:25.518231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ff793-04aa-4a7d-9465-62feb55f4bf4 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:25.518634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ff793-04aa-4a7d-9465-62feb55f4bf4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:25.518777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ff793-04aa-4a7d-9465-62feb55f4bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:25.579356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5e3ff793-04aa-4a7d-9465-62feb55f4bf4 None None] MFA Rules not processed for user `ad8825c294534a7c88877b48b5f8f07f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:57:25.580626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5e3ff793-04aa-4a7d-9465-62feb55f4bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:25.599369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:57:25.654491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 762/1518] 10.222.0.102 () {58 vars in 1020 bytes} [Wed Aug 24 04:57:25 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:57:29.778259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a6f9-83fd-455c-8349-8c009effdaed None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:29.778764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a6f9-83fd-455c-8349-8c009effdaed None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:29.778764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a6f9-83fd-455c-8349-8c009effdaed None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:29.780068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 757/1519] 10.222.0.105 () {54 vars in 966 bytes} [Wed Aug 24 04:57:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:57:29.784977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368250c-449b-49b0-9c98-f726ee9e2dd2 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:29.785431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368250c-449b-49b0-9c98-f726ee9e2dd2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:29.785557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368250c-449b-49b0-9c98-f726ee9e2dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:29.805034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-8368250c-449b-49b0-9c98-f726ee9e2dd2 None None] MFA Rules not processed for user `ad8825c294534a7c88877b48b5f8f07f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:57:29.806226 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8368250c-449b-49b0-9c98-f726ee9e2dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:29.814900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 763/1520] 10.222.0.105 () {58 vars in 1063 bytes} [Wed Aug 24 04:57:29 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:57:29.822833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6a6f9-83fd-455c-8349-8c009effdaed None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:57:29.836510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6a6f9-83fd-455c-8349-8c009effdaed None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:57:29.841286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b9c563-edd2-4b28-9fb8-4d9afa46d85a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:57:29.841861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9c563-edd2-4b28-9fb8-4d9afa46d85a service placement] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:29.842134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9c563-edd2-4b28-9fb8-4d9afa46d85a service placement] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:29.842401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b9c563-edd2-4b28-9fb8-4d9afa46d85a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:29.851049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9c563-edd2-4b28-9fb8-4d9afa46d85a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:57:29.852604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b9c563-edd2-4b28-9fb8-4d9afa46d85a service placement] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:57:29.861800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 758/1521] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:57:29 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:57:35.941047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868628c-1b8d-4cb2-8fb1-fabc0ffd391a None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:35.941581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868628c-1b8d-4cb2-8fb1-fabc0ffd391a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:35.941753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9868628c-1b8d-4cb2-8fb1-fabc0ffd391a None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:35.943721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:57:35 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:57:35.948684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36364bc3-8a6e-4f0c-ad65-c000516cebfb None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:35.949118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36364bc3-8a6e-4f0c-ad65-c000516cebfb None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:35.949227 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36364bc3-8a6e-4f0c-ad65-c000516cebfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:35.968284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-36364bc3-8a6e-4f0c-ad65-c000516cebfb None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:57:35.969473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-36364bc3-8a6e-4f0c-ad65-c000516cebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:35.977483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:57:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:57:36.133277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868628c-1b8d-4cb2-8fb1-fabc0ffd391a None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:57:36.147662 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9868628c-1b8d-4cb2-8fb1-fabc0ffd391a None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:57:36.153135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c8405-cfb0-4c34-b2df-5a6964a6c3b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:57:36.153722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8405-cfb0-4c34-b2df-5a6964a6c3b4 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:36.153999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8405-cfb0-4c34-b2df-5a6964a6c3b4 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:36.154270 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c8405-cfb0-4c34-b2df-5a6964a6c3b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:36.162980 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c8405-cfb0-4c34-b2df-5a6964a6c3b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:57:36.164570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c8405-cfb0-4c34-b2df-5a6964a6c3b4 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:57:36.174024 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 765/1524] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:57:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:57:37.014171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c0ec8-b72c-4fc3-b4c3-8d6190f977d7 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:37.014737 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c0ec8-b72c-4fc3-b4c3-8d6190f977d7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:37.014737 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c0ec8-b72c-4fc3-b4c3-8d6190f977d7 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:37.016111 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 04:57:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:57:37.021579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacdb2c-e67a-4ec7-b21e-dd500c27ad68 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:37.022008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacdb2c-e67a-4ec7-b21e-dd500c27ad68 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:37.022106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfacdb2c-e67a-4ec7-b21e-dd500c27ad68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:37.043090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-dfacdb2c-e67a-4ec7-b21e-dd500c27ad68 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:57:37.044318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-dfacdb2c-e67a-4ec7-b21e-dd500c27ad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:37.052422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 04:57:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:57:37.216603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c0ec8-b72c-4fc3-b4c3-8d6190f977d7 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:57:37.231225 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c0ec8-b72c-4fc3-b4c3-8d6190f977d7 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:57:37.236617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c043e6-673c-4ce5-9e77-56873bfcf583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:57:37.237223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c043e6-673c-4ce5-9e77-56873bfcf583 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:37.237483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c043e6-673c-4ce5-9e77-56873bfcf583 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:37.237753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c043e6-673c-4ce5-9e77-56873bfcf583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:37.246389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c043e6-673c-4ce5-9e77-56873bfcf583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:57:37.248025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c043e6-673c-4ce5-9e77-56873bfcf583 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:57:37.257828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 761/1527] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:57:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:57:41.222898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554904f-17e6-4269-b8a6-92ce3d7e91cd None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:41.226915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554904f-17e6-4269-b8a6-92ce3d7e91cd None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:41.226915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c554904f-17e6-4269-b8a6-92ce3d7e91cd None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:41.226915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:57:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:57:41.230042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a91f9-a807-48b6-a416-093cab7f2795 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:41.230440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a91f9-a807-48b6-a416-093cab7f2795 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:41.230599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a91f9-a807-48b6-a416-093cab7f2795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:41.250367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b16a91f9-a807-48b6-a416-093cab7f2795 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:57:41.251571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b16a91f9-a807-48b6-a416-093cab7f2795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:41.259967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:57:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:57:41.550453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554904f-17e6-4269-b8a6-92ce3d7e91cd None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:57:41.552197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c554904f-17e6-4269-b8a6-92ce3d7e91cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:41.628207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c554904f-17e6-4269-b8a6-92ce3d7e91cd None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:57:41.632275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8035c-316d-4c45-959b-2c5c9e23b66a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:57:41.632867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8035c-316d-4c45-959b-2c5c9e23b66a service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:41.633138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8035c-316d-4c45-959b-2c5c9e23b66a service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:41.633403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8035c-316d-4c45-959b-2c5c9e23b66a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:41.642077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8035c-316d-4c45-959b-2c5c9e23b66a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:57:41.643527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8035c-316d-4c45-959b-2c5c9e23b66a service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:57:41.653390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 768/1530] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:57:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:57:43.565876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965176-fdfa-4870-921d-03d285279729 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:43.566420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965176-fdfa-4870-921d-03d285279729 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:43.566420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965176-fdfa-4870-921d-03d285279729 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:43.568668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:57:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:57:43.573873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8244548-d5f2-41b7-9139-f454ee1ddb7e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:43.574291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8244548-d5f2-41b7-9139-f454ee1ddb7e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:43.574396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8244548-d5f2-41b7-9139-f454ee1ddb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:43.593255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-a8244548-d5f2-41b7-9139-f454ee1ddb7e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:57:43.594434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a8244548-d5f2-41b7-9139-f454ee1ddb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:43.603090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:57:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:57:43.893106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7965176-fdfa-4870-921d-03d285279729 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:57:43.904563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7965176-fdfa-4870-921d-03d285279729 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:57:43.908054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39372818-f891-46fd-8621-03d803c780f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:57:43.908643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39372818-f891-46fd-8621-03d803c780f7 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:43.908920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39372818-f891-46fd-8621-03d803c780f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:43.909192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39372818-f891-46fd-8621-03d803c780f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:43.918219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39372818-f891-46fd-8621-03d803c780f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:57:43.919736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39372818-f891-46fd-8621-03d803c780f7 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:57:43.929380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 764/1533] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:57:43 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:57:47.412180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec74f3-2755-4e51-a196-31caec2b1bf7 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:47.412739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec74f3-2755-4e51-a196-31caec2b1bf7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:47.412739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec74f3-2755-4e51-a196-31caec2b1bf7 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:47.414133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:57:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:57:47.419431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652618-e15c-45b6-a30a-c0ac6cc499eb None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:47.419873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652618-e15c-45b6-a30a-c0ac6cc499eb None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:47.419966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3652618-e15c-45b6-a30a-c0ac6cc499eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:47.440058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e3652618-e15c-45b6-a30a-c0ac6cc499eb None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:57:47.441306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e3652618-e15c-45b6-a30a-c0ac6cc499eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:47.449123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:57:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:57:48.008777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec74f3-2755-4e51-a196-31caec2b1bf7 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:57:48.010487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a0ec74f3-2755-4e51-a196-31caec2b1bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:48.076507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:57:48.090660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec74f3-2755-4e51-a196-31caec2b1bf7 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:57:48.094358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653472ad-3e6f-43b8-b6f8-00641e64ca4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:57:48.094959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653472ad-3e6f-43b8-b6f8-00641e64ca4a service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:48.095281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653472ad-3e6f-43b8-b6f8-00641e64ca4a service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:48.095536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653472ad-3e6f-43b8-b6f8-00641e64ca4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:48.104619 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653472ad-3e6f-43b8-b6f8-00641e64ca4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:57:48.106038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653472ad-3e6f-43b8-b6f8-00641e64ca4a service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:57:48.115521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 771/1536] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:57:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:57:50.095405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1730d-8c60-429d-bfb5-f778dd2d9acb None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:50.095984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1730d-8c60-429d-bfb5-f778dd2d9acb None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:50.095984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1730d-8c60-429d-bfb5-f778dd2d9acb None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:50.097305 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 04:57:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:57:50.103470 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77d83f-be2a-4835-a5bb-f3be9d92aa3e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:50.103903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77d83f-be2a-4835-a5bb-f3be9d92aa3e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:50.104157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77d83f-be2a-4835-a5bb-f3be9d92aa3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:50.123302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5d77d83f-be2a-4835-a5bb-f3be9d92aa3e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:57:50.127130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5d77d83f-be2a-4835-a5bb-f3be9d92aa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:50.132628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 04:57:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:57:50.687000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1730d-8c60-429d-bfb5-f778dd2d9acb None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:57:50.688730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-84f1730d-8c60-429d-bfb5-f778dd2d9acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:57:50.696947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=84 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:57:50.702320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1730d-8c60-429d-bfb5-f778dd2d9acb None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:57:50.705779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd548433-a497-4032-8a2b-1ea4bb3ed3cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:57:50.715869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd548433-a497-4032-8a2b-1ea4bb3ed3cd service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:57:50.715869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd548433-a497-4032-8a2b-1ea4bb3ed3cd service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:57:50.715869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd548433-a497-4032-8a2b-1ea4bb3ed3cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:57:50.716151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd548433-a497-4032-8a2b-1ea4bb3ed3cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:57:50.717556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd548433-a497-4032-8a2b-1ea4bb3ed3cd service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:57:50.726896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 767/1539] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:57:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:58:00.118685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=11 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:58:00.742044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:14.655855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880885a4-881e-4610-ba7b-d05027ba96ff None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:14.656520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880885a4-881e-4610-ba7b-d05027ba96ff None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:14.656520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880885a4-881e-4610-ba7b-d05027ba96ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:14.684388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-880885a4-881e-4610-ba7b-d05027ba96ff None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:14.685914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-880885a4-881e-4610-ba7b-d05027ba96ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:14.700305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 773/1540] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:14.709517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9cb68-5287-45e3-afe5-6c31c5d0c918 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:14.710152 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9cb68-5287-45e3-afe5-6c31c5d0c918 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:14.710152 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9cb68-5287-45e3-afe5-6c31c5d0c918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:14.748911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b1e9cb68-5287-45e3-afe5-6c31c5d0c918 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:14.750155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b1e9cb68-5287-45e3-afe5-6c31c5d0c918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:14.759594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 768/1541] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:14.764570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880885a4-881e-4610-ba7b-d05027ba96ff None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:14.775925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880885a4-881e-4610-ba7b-d05027ba96ff None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:14.786816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a916f87-a40a-4a00-b86d-c87a78210df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:14.787479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a916f87-a40a-4a00-b86d-c87a78210df4 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:14.787758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a916f87-a40a-4a00-b86d-c87a78210df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:14.788028 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a916f87-a40a-4a00-b86d-c87a78210df4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:14.788689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a916f87-a40a-4a00-b86d-c87a78210df4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:59:14.788987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a916f87-a40a-4a00-b86d-c87a78210df4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:14.790680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a916f87-a40a-4a00-b86d-c87a78210df4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:14.808482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 774/1542] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:59:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:59:14.816976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9cb68-5287-45e3-afe5-6c31c5d0c918 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:14.827716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9cb68-5287-45e3-afe5-6c31c5d0c918 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:14.838468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872ef069-cc5b-4dae-8172-1cc147fc2748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:14.839129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ef069-cc5b-4dae-8172-1cc147fc2748 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:14.839406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ef069-cc5b-4dae-8172-1cc147fc2748 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:14.839674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ef069-cc5b-4dae-8172-1cc147fc2748 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:14.840317 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ef069-cc5b-4dae-8172-1cc147fc2748 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:14.842680 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872ef069-cc5b-4dae-8172-1cc147fc2748 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:14.862844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895a082-cead-46ce-bf96-82357066cfe5 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:14.863533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895a082-cead-46ce-bf96-82357066cfe5 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:14.863812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895a082-cead-46ce-bf96-82357066cfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:14.904106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f895a082-cead-46ce-bf96-82357066cfe5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:14.905748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f895a082-cead-46ce-bf96-82357066cfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:14.916314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 775/1543] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:14.917449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5c3f1-6841-460e-9133-b7537d0cad11 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:14.917739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5c3f1-6841-460e-9133-b7537d0cad11 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:14.918006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5c3f1-6841-460e-9133-b7537d0cad11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:14.940544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-55e5c3f1-6841-460e-9133-b7537d0cad11 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:14.941804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-55e5c3f1-6841-460e-9133-b7537d0cad11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:14.958110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 776/1544] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:14.961056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1006ca-a88c-47f0-a3c6-02ceaf6e026c None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:14.961056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1006ca-a88c-47f0-a3c6-02ceaf6e026c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:14.961056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1006ca-a88c-47f0-a3c6-02ceaf6e026c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:14.990313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 769/1545] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:14 2022] POST /identity/v3/projects => generated 386 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:14.991748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df18fe1-2344-431f-a23d-fd645857489e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:14.992033 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df18fe1-2344-431f-a23d-fd645857489e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:14.992307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df18fe1-2344-431f-a23d-fd645857489e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:14.997049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ed1006ca-a88c-47f0-a3c6-02ceaf6e026c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:14.998493 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ed1006ca-a88c-47f0-a3c6-02ceaf6e026c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.032100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2df18fe1-2344-431f-a23d-fd645857489e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.033323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2df18fe1-2344-431f-a23d-fd645857489e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.079050 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 777/1546] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.080402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebd41d-45b5-4272-b966-923bfde1e866 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.080688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebd41d-45b5-4272-b966-923bfde1e866 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.081088 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebd41d-45b5-4272-b966-923bfde1e866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.101262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-8cebd41d-45b5-4272-b966-923bfde1e866 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.102499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8cebd41d-45b5-4272-b966-923bfde1e866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.104706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 770/1547] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.105596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df18fe1-2344-431f-a23d-fd645857489e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.110398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 778/1548] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.111603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31537aa-13ee-43e6-8bdc-9b9fda7dcd03 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.111889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31537aa-13ee-43e6-8bdc-9b9fda7dcd03 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.112153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31537aa-13ee-43e6-8bdc-9b9fda7dcd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.117735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df18fe1-2344-431f-a23d-fd645857489e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.122766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da6c42-3d73-4287-9b4c-270453dc86a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.123398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6c42-3d73-4287-9b4c-270453dc86a1 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.123674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6c42-3d73-4287-9b4c-270453dc86a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.123945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da6c42-3d73-4287-9b4c-270453dc86a1 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.124568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da6c42-3d73-4287-9b4c-270453dc86a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.125578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: WARNING py.warnings [None req-00da6c42-3d73-4287-9b4c-270453dc86a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:59:15.125578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: warnings.warn(msg) Aug 24 04:59:15.125578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: Aug 24 04:59:15.126769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00da6c42-3d73-4287-9b4c-270453dc86a1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.131425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c31537aa-13ee-43e6-8bdc-9b9fda7dcd03 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.132653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c31537aa-13ee-43e6-8bdc-9b9fda7dcd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.141694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 779/1549] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.143103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6b604-a992-4b67-b116-cd5ac6755fb7 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.143391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6b604-a992-4b67-b116-cd5ac6755fb7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.143665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6b604-a992-4b67-b116-cd5ac6755fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.163384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-92b6b604-a992-4b67-b116-cd5ac6755fb7 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.164606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-92b6b604-a992-4b67-b116-cd5ac6755fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.173024 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 780/1550] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.175008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a483a4-757c-4c34-a39e-c988f5bf1107 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.175479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a483a4-757c-4c34-a39e-c988f5bf1107 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.175756 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a483a4-757c-4c34-a39e-c988f5bf1107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.196255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f4a483a4-757c-4c34-a39e-c988f5bf1107 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.197462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f4a483a4-757c-4c34-a39e-c988f5bf1107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.205576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 781/1551] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.206866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ad5b-45be-425e-b938-bc9dcfbc81a2 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.207199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ad5b-45be-425e-b938-bc9dcfbc81a2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.207458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b692ad5b-45be-425e-b938-bc9dcfbc81a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.215339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 771/1552] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:15.216069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da6c42-3d73-4287-9b4c-270453dc86a1 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.225516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da6c42-3d73-4287-9b4c-270453dc86a1 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.229747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b692ad5b-45be-425e-b938-bc9dcfbc81a2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.229747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b692ad5b-45be-425e-b938-bc9dcfbc81a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.230660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74993d-3e56-4c2a-9d35-afc7ba709c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.231287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74993d-3e56-4c2a-9d35-afc7ba709c3a admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.231564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74993d-3e56-4c2a-9d35-afc7ba709c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.231833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74993d-3e56-4c2a-9d35-afc7ba709c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.232467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74993d-3e56-4c2a-9d35-afc7ba709c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:59:15.232764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74993d-3e56-4c2a-9d35-afc7ba709c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.233777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: WARNING py.warnings [None req-4b74993d-3e56-4c2a-9d35-afc7ba709c3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:59:15.233777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: warnings.warn(msg) Aug 24 04:59:15.233777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: Aug 24 04:59:15.234648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74993d-3e56-4c2a-9d35-afc7ba709c3a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.237644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 782/1553] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.238681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692ad5b-45be-425e-b938-bc9dcfbc81a2 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.248376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b692ad5b-45be-425e-b938-bc9dcfbc81a2 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.250705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 772/1554] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:59:15.252239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8b12e-f60d-4815-af7e-9612aa8c14b3 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.252518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8b12e-f60d-4815-af7e-9612aa8c14b3 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.252793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8b12e-f60d-4815-af7e-9612aa8c14b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.260996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20ab66-0b35-4f41-bb9d-84be174bd946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.261612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20ab66-0b35-4f41-bb9d-84be174bd946 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.261891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20ab66-0b35-4f41-bb9d-84be174bd946 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.262160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20ab66-0b35-4f41-bb9d-84be174bd946 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.262809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20ab66-0b35-4f41-bb9d-84be174bd946 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:59:15.263143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20ab66-0b35-4f41-bb9d-84be174bd946 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.264767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20ab66-0b35-4f41-bb9d-84be174bd946 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.273301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-49d8b12e-f60d-4815-af7e-9612aa8c14b3 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.275242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-49d8b12e-f60d-4815-af7e-9612aa8c14b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.277770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 783/1555] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:59:15.278967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb4498-2eb2-454b-9f57-e30279723a9c None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.279288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb4498-2eb2-454b-9f57-e30279723a9c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.279586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb4498-2eb2-454b-9f57-e30279723a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.284363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 773/1556] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.288385 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab2875-4445-4e02-a034-c4de3fdaee1e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.288654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab2875-4445-4e02-a034-c4de3fdaee1e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.288927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab2875-4445-4e02-a034-c4de3fdaee1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.301534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-9abb4498-2eb2-454b-9f57-e30279723a9c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.302732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9abb4498-2eb2-454b-9f57-e30279723a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.308811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-08ab2875-4445-4e02-a034-c4de3fdaee1e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.315129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-08ab2875-4445-4e02-a034-c4de3fdaee1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.315251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 784/1557] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.315251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573168fe-0782-4891-8b0f-00bc1b6f6cb0 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.315251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573168fe-0782-4891-8b0f-00bc1b6f6cb0 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.315251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573168fe-0782-4891-8b0f-00bc1b6f6cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.319854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 774/1558] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.320545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab2875-4445-4e02-a034-c4de3fdaee1e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.330911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab2875-4445-4e02-a034-c4de3fdaee1e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.335708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc45c87-0e19-4966-be51-fd909b49ed34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.336958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc45c87-0e19-4966-be51-fd909b49ed34 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.337274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc45c87-0e19-4966-be51-fd909b49ed34 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.337548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc45c87-0e19-4966-be51-fd909b49ed34 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.338180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc45c87-0e19-4966-be51-fd909b49ed34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.338386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-573168fe-0782-4891-8b0f-00bc1b6f6cb0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.339518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-573168fe-0782-4891-8b0f-00bc1b6f6cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.339799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc45c87-0e19-4966-be51-fd909b49ed34 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.347856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 785/1559] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.348849 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6bc02-41e6-40e1-bde9-48bf81753562 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.349131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6bc02-41e6-40e1-bde9-48bf81753562 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.349402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6bc02-41e6-40e1-bde9-48bf81753562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.350771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 775/1560] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:15 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:15.351687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc45c87-0e19-4966-be51-fd909b49ed34 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.361600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc45c87-0e19-4966-be51-fd909b49ed34 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.366915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9035aa9-f00f-4fc4-bc08-408904872aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.367594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9035aa9-f00f-4fc4-bc08-408904872aed admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.367882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9035aa9-f00f-4fc4-bc08-408904872aed admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.368200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9035aa9-f00f-4fc4-bc08-408904872aed admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.368824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9035aa9-f00f-4fc4-bc08-408904872aed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.370735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9035aa9-f00f-4fc4-bc08-408904872aed admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.371121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-53a6bc02-41e6-40e1-bde9-48bf81753562 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:15.372311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-53a6bc02-41e6-40e1-bde9-48bf81753562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:15.380195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 786/1561] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:15.380955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6bc02-41e6-40e1-bde9-48bf81753562 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.393838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a6bc02-41e6-40e1-bde9-48bf81753562 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.398409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3b25f-bf2e-4470-b655-affeb434c6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.399935 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3b25f-bf2e-4470-b655-affeb434c6db admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.400384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3b25f-bf2e-4470-b655-affeb434c6db admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.406732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee3b25f-bf2e-4470-b655-affeb434c6db admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.406732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3b25f-bf2e-4470-b655-affeb434c6db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.406732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WARNING py.warnings [None req-9ee3b25f-bf2e-4470-b655-affeb434c6db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:59:15.406732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: warnings.warn(msg) Aug 24 04:59:15.406732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: Aug 24 04:59:15.406732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee3b25f-bf2e-4470-b655-affeb434c6db admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.480920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 776/1562] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/projects => generated 382 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:15.480920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9035aa9-f00f-4fc4-bc08-408904872aed admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.525730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:15.539114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9035aa9-f00f-4fc4-bc08-408904872aed admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.550729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c109b5b-aa26-4c00-8aa8-7f3e88f2839b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.550729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c109b5b-aa26-4c00-8aa8-7f3e88f2839b admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.550729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c109b5b-aa26-4c00-8aa8-7f3e88f2839b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.550729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c109b5b-aa26-4c00-8aa8-7f3e88f2839b admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.551159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c109b5b-aa26-4c00-8aa8-7f3e88f2839b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:59:15.551159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c109b5b-aa26-4c00-8aa8-7f3e88f2839b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.551328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 787/1563] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/projects => generated 380 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:15.551433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c109b5b-aa26-4c00-8aa8-7f3e88f2839b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.551535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3b25f-bf2e-4470-b655-affeb434c6db admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.560366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 777/1564] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:59:15.568801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c109b5b-aa26-4c00-8aa8-7f3e88f2839b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.580975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c109b5b-aa26-4c00-8aa8-7f3e88f2839b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.594612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:15.613269 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee3b25f-bf2e-4470-b655-affeb434c6db admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.618570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78292e73-ae13-41eb-b8b3-d952b165d054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.619210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78292e73-ae13-41eb-b8b3-d952b165d054 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.619489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78292e73-ae13-41eb-b8b3-d952b165d054 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.619763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78292e73-ae13-41eb-b8b3-d952b165d054 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.620395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78292e73-ae13-41eb-b8b3-d952b165d054 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:59:15.620688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78292e73-ae13-41eb-b8b3-d952b165d054 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.622268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78292e73-ae13-41eb-b8b3-d952b165d054 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.634313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 788/1565] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:59:15.635122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78292e73-ae13-41eb-b8b3-d952b165d054 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.644047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9482682-db9c-4ac2-a369-f7cd4b4ea5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.644665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9482682-db9c-4ac2-a369-f7cd4b4ea5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.644773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78292e73-ae13-41eb-b8b3-d952b165d054 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.644933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9482682-db9c-4ac2-a369-f7cd4b4ea5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.645209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9482682-db9c-4ac2-a369-f7cd4b4ea5c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.645848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9482682-db9c-4ac2-a369-f7cd4b4ea5c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:59:15.646176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9482682-db9c-4ac2-a369-f7cd4b4ea5c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.647842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9482682-db9c-4ac2-a369-f7cd4b4ea5c9 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.649449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3577ae5-1144-4d75-a81a-6211d7ccf407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.650042 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3577ae5-1144-4d75-a81a-6211d7ccf407 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.650319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3577ae5-1144-4d75-a81a-6211d7ccf407 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.650589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3577ae5-1144-4d75-a81a-6211d7ccf407 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.651241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3577ae5-1144-4d75-a81a-6211d7ccf407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:59:15.651542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3577ae5-1144-4d75-a81a-6211d7ccf407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.653093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3577ae5-1144-4d75-a81a-6211d7ccf407 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.659962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 778/1566] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:59:15.660639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9482682-db9c-4ac2-a369-f7cd4b4ea5c9 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.665642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 789/1567] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:59:15.667001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3577ae5-1144-4d75-a81a-6211d7ccf407 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.677766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9482682-db9c-4ac2-a369-f7cd4b4ea5c9 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.680580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3577ae5-1144-4d75-a81a-6211d7ccf407 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.683498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9cb0f-f4c9-4e65-833f-5ffeec44196b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.684110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9cb0f-f4c9-4e65-833f-5ffeec44196b admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.684388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9cb0f-f4c9-4e65-833f-5ffeec44196b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.684662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9cb0f-f4c9-4e65-833f-5ffeec44196b admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.685318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9cb0f-f4c9-4e65-833f-5ffeec44196b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.685432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ec76e-c66b-44c4-872c-6432430ca9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.686230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ec76e-c66b-44c4-872c-6432430ca9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.686514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ec76e-c66b-44c4-872c-6432430ca9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.686791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0ec76e-c66b-44c4-872c-6432430ca9d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.687641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ec76e-c66b-44c4-872c-6432430ca9d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:59:15.687873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9cb0f-f4c9-4e65-833f-5ffeec44196b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.688086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ec76e-c66b-44c4-872c-6432430ca9d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.689686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0ec76e-c66b-44c4-872c-6432430ca9d9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.699604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 779/1568] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:15 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:15.701195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9cb0f-f4c9-4e65-833f-5ffeec44196b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.708423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 790/1569] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:59:15.708423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ec76e-c66b-44c4-872c-6432430ca9d9 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.713628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9cb0f-f4c9-4e65-833f-5ffeec44196b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.718085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0ec76e-c66b-44c4-872c-6432430ca9d9 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.718321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f2c71-46a6-48a8-bf7c-9f196f2625a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.718915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f2c71-46a6-48a8-bf7c-9f196f2625a0 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.719221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f2c71-46a6-48a8-bf7c-9f196f2625a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.719496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f2c71-46a6-48a8-bf7c-9f196f2625a0 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.720141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f2c71-46a6-48a8-bf7c-9f196f2625a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.722010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071f2c71-46a6-48a8-bf7c-9f196f2625a0 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.723396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453ea84-bf3b-40fd-8d5f-53c9b98ba06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.723981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453ea84-bf3b-40fd-8d5f-53c9b98ba06e admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.724316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453ea84-bf3b-40fd-8d5f-53c9b98ba06e admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.724522 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453ea84-bf3b-40fd-8d5f-53c9b98ba06e admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.725129 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453ea84-bf3b-40fd-8d5f-53c9b98ba06e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.727045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453ea84-bf3b-40fd-8d5f-53c9b98ba06e admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.829725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 780/1570] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:15.830527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f2c71-46a6-48a8-bf7c-9f196f2625a0 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.840496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f2c71-46a6-48a8-bf7c-9f196f2625a0 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.843207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 791/1571] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:15.844117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453ea84-bf3b-40fd-8d5f-53c9b98ba06e admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.848046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79235ff1-a2e8-4ea6-bc67-0f29531946d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.848796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79235ff1-a2e8-4ea6-bc67-0f29531946d7 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.849191 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79235ff1-a2e8-4ea6-bc67-0f29531946d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.849468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79235ff1-a2e8-4ea6-bc67-0f29531946d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.850114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79235ff1-a2e8-4ea6-bc67-0f29531946d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.852922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79235ff1-a2e8-4ea6-bc67-0f29531946d7 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.856191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453ea84-bf3b-40fd-8d5f-53c9b98ba06e admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:15.860908 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae67ff9-36ee-4234-81c1-caa4725ed575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:15.861492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67ff9-36ee-4234-81c1-caa4725ed575 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:15.871444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67ff9-36ee-4234-81c1-caa4725ed575 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:15.871444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67ff9-36ee-4234-81c1-caa4725ed575 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:15.871444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae67ff9-36ee-4234-81c1-caa4725ed575 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:15.871444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae67ff9-36ee-4234-81c1-caa4725ed575 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:15.946820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 781/1572] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/projects => generated 388 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:15.947695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79235ff1-a2e8-4ea6-bc67-0f29531946d7 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:15.979428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 792/1573] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/projects => generated 374 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:15.979428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae67ff9-36ee-4234-81c1-caa4725ed575 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.016694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79235ff1-a2e8-4ea6-bc67-0f29531946d7 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.041766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae67ff9-36ee-4234-81c1-caa4725ed575 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.047674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577797e9-44eb-48d9-aff5-b60a0f68e8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.047674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577797e9-44eb-48d9-aff5-b60a0f68e8ce admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.047674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577797e9-44eb-48d9-aff5-b60a0f68e8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.047990 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:16.048108 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577797e9-44eb-48d9-aff5-b60a0f68e8ce admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.048212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577797e9-44eb-48d9-aff5-b60a0f68e8ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.050016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577797e9-44eb-48d9-aff5-b60a0f68e8ce admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.075971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9087c95-89e1-42a2-9665-0c9dfb5ceed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.076573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087c95-89e1-42a2-9665-0c9dfb5ceed1 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.076845 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087c95-89e1-42a2-9665-0c9dfb5ceed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.077163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9087c95-89e1-42a2-9665-0c9dfb5ceed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.077807 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9087c95-89e1-42a2-9665-0c9dfb5ceed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.079750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9087c95-89e1-42a2-9665-0c9dfb5ceed1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.187540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 793/1574] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/projects => generated 376 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:16.188328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577797e9-44eb-48d9-aff5-b60a0f68e8ce admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.199408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 782/1575] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:15 2022] POST /identity/v3/projects => generated 394 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:16.202456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9087c95-89e1-42a2-9665-0c9dfb5ceed1 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.246663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:16.257050 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577797e9-44eb-48d9-aff5-b60a0f68e8ce admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.261929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029e624-5b2f-4de1-b276-32178798b515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.262404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9087c95-89e1-42a2-9665-0c9dfb5ceed1 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.262585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029e624-5b2f-4de1-b276-32178798b515 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.262863 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029e624-5b2f-4de1-b276-32178798b515 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.263166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029e624-5b2f-4de1-b276-32178798b515 admin admin] PATH_INFO: `/v3/projects/904b8da68db84c3ea1f8785c08ecdc58/users/00e30c4bef674af481586297243901db/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.267231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1d4e1-f403-49b0-9f8e-25ea344b1ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.267816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1d4e1-f403-49b0-9f8e-25ea344b1ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.268097 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1d4e1-f403-49b0-9f8e-25ea344b1ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.268368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1d4e1-f403-49b0-9f8e-25ea344b1ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.268986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1d4e1-f403-49b0-9f8e-25ea344b1ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.270989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1d4e1-f403-49b0-9f8e-25ea344b1ab3 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.279476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029e624-5b2f-4de1-b276-32178798b515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=904b8da68db84c3ea1f8785c08ecdc58, user_id=00e30c4bef674af481586297243901db, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.280602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WARNING py.warnings [None req-a029e624-5b2f-4de1-b276-32178798b515 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:59:16.280602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: warnings.warn(msg) Aug 24 04:59:16.280602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: Aug 24 04:59:16.282536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029e624-5b2f-4de1-b276-32178798b515 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.402780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 794/1576] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:16 2022] PUT /identity/v3/projects/904b8da68db84c3ea1f8785c08ecdc58/users/00e30c4bef674af481586297243901db/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:16.403783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029e624-5b2f-4de1-b276-32178798b515 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.435661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 783/1577] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:16 2022] POST /identity/v3/projects => generated 386 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:16.435661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1d4e1-f403-49b0-9f8e-25ea344b1ab3 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.472848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029e624-5b2f-4de1-b276-32178798b515 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.498161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1d4e1-f403-49b0-9f8e-25ea344b1ab3 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.502856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c13ec-5920-4637-a47c-7f4670a8dacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.503506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c13ec-5920-4637-a47c-7f4670a8dacb admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.503783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c13ec-5920-4637-a47c-7f4670a8dacb admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.504054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c13ec-5920-4637-a47c-7f4670a8dacb admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.504710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c13ec-5920-4637-a47c-7f4670a8dacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.506346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4c13ec-5920-4637-a47c-7f4670a8dacb admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.520408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 784/1578] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:16 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:16.520408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c13ec-5920-4637-a47c-7f4670a8dacb admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.531120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343566d-5d89-4434-bd89-9cafd0161e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.531325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c13ec-5920-4637-a47c-7f4670a8dacb admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.531759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343566d-5d89-4434-bd89-9cafd0161e34 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.532036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343566d-5d89-4434-bd89-9cafd0161e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.532423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343566d-5d89-4434-bd89-9cafd0161e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.533058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343566d-5d89-4434-bd89-9cafd0161e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.534664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343566d-5d89-4434-bd89-9cafd0161e34 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.535902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c5de1-f789-497c-8f23-b63ad02c39f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.536777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c5de1-f789-497c-8f23-b63ad02c39f7 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.536777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c5de1-f789-497c-8f23-b63ad02c39f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.537036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c5de1-f789-497c-8f23-b63ad02c39f7 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.537656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c5de1-f789-497c-8f23-b63ad02c39f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.540362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c5de1-f789-497c-8f23-b63ad02c39f7 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.549186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 795/1579] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:16 2022] GET /identity/v3/roles => generated 1427 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:16.549186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343566d-5d89-4434-bd89-9cafd0161e34 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.560961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343566d-5d89-4434-bd89-9cafd0161e34 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.562406 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d452e-de3d-43e6-a8d2-80b73e1d4476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.562992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d452e-de3d-43e6-a8d2-80b73e1d4476 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.563307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d452e-de3d-43e6-a8d2-80b73e1d4476 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.563580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d452e-de3d-43e6-a8d2-80b73e1d4476 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.564220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d452e-de3d-43e6-a8d2-80b73e1d4476 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.566123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d452e-de3d-43e6-a8d2-80b73e1d4476 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.710440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 785/1580] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:16 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:16.712097 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c5de1-f789-497c-8f23-b63ad02c39f7 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.721648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c5de1-f789-497c-8f23-b63ad02c39f7 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.726178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225592e-65d7-47b8-91e2-5bd4fc431356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.726848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225592e-65d7-47b8-91e2-5bd4fc431356 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.727187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225592e-65d7-47b8-91e2-5bd4fc431356 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.727467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225592e-65d7-47b8-91e2-5bd4fc431356 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.728061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225592e-65d7-47b8-91e2-5bd4fc431356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.729948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225592e-65d7-47b8-91e2-5bd4fc431356 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.739172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 796/1581] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:16 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:16.739963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d452e-de3d-43e6-a8d2-80b73e1d4476 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.750235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d452e-de3d-43e6-a8d2-80b73e1d4476 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.754895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa093b09-b8e4-4362-83bc-2e87d75f6aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.755513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093b09-b8e4-4362-83bc-2e87d75f6aac admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.755789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093b09-b8e4-4362-83bc-2e87d75f6aac admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.756058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa093b09-b8e4-4362-83bc-2e87d75f6aac admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.756667 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa093b09-b8e4-4362-83bc-2e87d75f6aac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.759249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa093b09-b8e4-4362-83bc-2e87d75f6aac admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.847673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 786/1582] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:16 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:16.849003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43132e4-b919-4bdd-a485-403edfc4ac40 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.849425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43132e4-b919-4bdd-a485-403edfc4ac40 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.849690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43132e4-b919-4bdd-a485-403edfc4ac40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.856889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:16.877207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e43132e4-b919-4bdd-a485-403edfc4ac40 None None] MFA Rules not processed for user `00e30c4bef674af481586297243901db`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:16.883536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 797/1583] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:16 2022] POST /identity/v3/users => generated 298 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:16.883536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa093b09-b8e4-4362-83bc-2e87d75f6aac admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.883763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e43132e4-b919-4bdd-a485-403edfc4ac40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:16.890011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa093b09-b8e4-4362-83bc-2e87d75f6aac admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.894869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80571611-3f69-40fb-a124-8a1242d2d025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.895506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80571611-3f69-40fb-a124-8a1242d2d025 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.895779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80571611-3f69-40fb-a124-8a1242d2d025 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.896047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80571611-3f69-40fb-a124-8a1242d2d025 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.896684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80571611-3f69-40fb-a124-8a1242d2d025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.898603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80571611-3f69-40fb-a124-8a1242d2d025 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.943726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 787/1584] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:16 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:16.944589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43132e4-b919-4bdd-a485-403edfc4ac40 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.955779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43132e4-b919-4bdd-a485-403edfc4ac40 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.960422 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d3a91-4790-4920-92c2-0e343298bd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.961046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d3a91-4790-4920-92c2-0e343298bd52 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.961458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d3a91-4790-4920-92c2-0e343298bd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.961777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d3a91-4790-4920-92c2-0e343298bd52 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.962412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d3a91-4790-4920-92c2-0e343298bd52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.964055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d3a91-4790-4920-92c2-0e343298bd52 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:16.974855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 788/1585] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:16 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:16.978772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d3a91-4790-4920-92c2-0e343298bd52 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:16.985418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d3a91-4790-4920-92c2-0e343298bd52 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:16.990112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1a1dc-1329-4b5e-83aa-4ae2c899c95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:16.990699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1a1dc-1329-4b5e-83aa-4ae2c899c95d admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:16.990977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1a1dc-1329-4b5e-83aa-4ae2c899c95d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:16.991279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1a1dc-1329-4b5e-83aa-4ae2c899c95d admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:16.991933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1a1dc-1329-4b5e-83aa-4ae2c899c95d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:16.994255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1a1dc-1329-4b5e-83aa-4ae2c899c95d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.004063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 789/1586] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:16 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.004911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1a1dc-1329-4b5e-83aa-4ae2c899c95d admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.012976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 798/1587] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:16 2022] POST /identity/v3/users => generated 294 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:17.012976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80571611-3f69-40fb-a124-8a1242d2d025 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.016249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1a1dc-1329-4b5e-83aa-4ae2c899c95d admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.021023 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de77f94-7f72-4d4d-b588-8c49aeeaf6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.021630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de77f94-7f72-4d4d-b588-8c49aeeaf6fb admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.021894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de77f94-7f72-4d4d-b588-8c49aeeaf6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.022171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de77f94-7f72-4d4d-b588-8c49aeeaf6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.022826 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de77f94-7f72-4d4d-b588-8c49aeeaf6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.023307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80571611-3f69-40fb-a124-8a1242d2d025 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.024525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de77f94-7f72-4d4d-b588-8c49aeeaf6fb admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.028064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5101b-c9e9-4ac6-a465-c25229e402b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.028657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5101b-c9e9-4ac6-a465-c25229e402b3 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.028936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5101b-c9e9-4ac6-a465-c25229e402b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.029253 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5101b-c9e9-4ac6-a465-c25229e402b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.029911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5101b-c9e9-4ac6-a465-c25229e402b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.031544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5101b-c9e9-4ac6-a465-c25229e402b3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.035992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 790/1588] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.036833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de77f94-7f72-4d4d-b588-8c49aeeaf6fb admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.045508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 799/1589] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.045508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5101b-c9e9-4ac6-a465-c25229e402b3 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.049571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de77f94-7f72-4d4d-b588-8c49aeeaf6fb admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.054489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1307c409-9b5c-4ed3-8a71-af46b654c8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.055482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5101b-c9e9-4ac6-a465-c25229e402b3 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.055802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307c409-9b5c-4ed3-8a71-af46b654c8ec admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.056083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307c409-9b5c-4ed3-8a71-af46b654c8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.056356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1307c409-9b5c-4ed3-8a71-af46b654c8ec admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.057015 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307c409-9b5c-4ed3-8a71-af46b654c8ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.058710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1307c409-9b5c-4ed3-8a71-af46b654c8ec admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.060907 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a554468-fd25-4ff3-92d0-60a4bcb7362c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.061510 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a554468-fd25-4ff3-92d0-60a4bcb7362c admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.061784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a554468-fd25-4ff3-92d0-60a4bcb7362c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.067070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a554468-fd25-4ff3-92d0-60a4bcb7362c admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.067070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a554468-fd25-4ff3-92d0-60a4bcb7362c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.067070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a554468-fd25-4ff3-92d0-60a4bcb7362c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.073888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 791/1590] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.073888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a632de-112c-41ff-8a5f-a444c0937f47 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.073888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a632de-112c-41ff-8a5f-a444c0937f47 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.073888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a632de-112c-41ff-8a5f-a444c0937f47 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.074525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 792/1591] 10.222.0.105 () {54 vars in 967 bytes} [Wed Aug 24 04:59:17 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:59:17.075208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a632de-112c-41ff-8a5f-a444c0937f47 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.075884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 800/1592] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.076612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a554468-fd25-4ff3-92d0-60a4bcb7362c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.084402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:17.089127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a554468-fd25-4ff3-92d0-60a4bcb7362c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.091539 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a632de-112c-41ff-8a5f-a444c0937f47 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.094753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb163b-1b4f-495e-b2bb-d8aa051f774b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.095625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb163b-1b4f-495e-b2bb-d8aa051f774b admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.099965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb163b-1b4f-495e-b2bb-d8aa051f774b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.099965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb163b-1b4f-495e-b2bb-d8aa051f774b admin admin] PATH_INFO: `/v3/projects/46bf86c3f1894b9484ff64f8a76382fa/users/c13efb9d7b98426dba25a64c9c15b6e8/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.100163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef886e7f-136f-4455-8b55-b583aec1ec91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.100163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef886e7f-136f-4455-8b55-b583aec1ec91 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.100163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef886e7f-136f-4455-8b55-b583aec1ec91 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.100163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef886e7f-136f-4455-8b55-b583aec1ec91 admin admin] PATH_INFO: `/v3/projects/ae5545057ecc4c02817cd846dc73b55b/users/312b0174466b4b00be19d78bdc01fdb4/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.111124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb163b-1b4f-495e-b2bb-d8aa051f774b admin admin] RBAC: Authorizing `identity:create_grant(project_id=46bf86c3f1894b9484ff64f8a76382fa, user_id=c13efb9d7b98426dba25a64c9c15b6e8, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.112592 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef886e7f-136f-4455-8b55-b583aec1ec91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae5545057ecc4c02817cd846dc73b55b, user_id=312b0174466b4b00be19d78bdc01fdb4, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.113826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb163b-1b4f-495e-b2bb-d8aa051f774b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.115265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef886e7f-136f-4455-8b55-b583aec1ec91 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.214266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 793/1593] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:17 2022] PUT /identity/v3/projects/ae5545057ecc4c02817cd846dc73b55b/users/312b0174466b4b00be19d78bdc01fdb4/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:17.220121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 801/1594] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:17 2022] PUT /identity/v3/projects/46bf86c3f1894b9484ff64f8a76382fa/users/c13efb9d7b98426dba25a64c9c15b6e8/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:17.220311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef886e7f-136f-4455-8b55-b583aec1ec91 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.220407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb163b-1b4f-495e-b2bb-d8aa051f774b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.278232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef886e7f-136f-4455-8b55-b583aec1ec91 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.289754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb163b-1b4f-495e-b2bb-d8aa051f774b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.294718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decefc2d-c40b-4ee3-81c6-d45b6cc42e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.295351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decefc2d-c40b-4ee3-81c6-d45b6cc42e7a admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.295623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decefc2d-c40b-4ee3-81c6-d45b6cc42e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.302002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decefc2d-c40b-4ee3-81c6-d45b6cc42e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.302002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decefc2d-c40b-4ee3-81c6-d45b6cc42e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.302002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decefc2d-c40b-4ee3-81c6-d45b6cc42e7a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.308470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:17.309644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 802/1595] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/roles => generated 1427 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.309990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decefc2d-c40b-4ee3-81c6-d45b6cc42e7a admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.322323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decefc2d-c40b-4ee3-81c6-d45b6cc42e7a admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.327111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e4179-9918-4ce3-b93d-b3761de20b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.327687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e4179-9918-4ce3-b93d-b3761de20b44 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.327968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e4179-9918-4ce3-b93d-b3761de20b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.328229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e4179-9918-4ce3-b93d-b3761de20b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.328849 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e4179-9918-4ce3-b93d-b3761de20b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.330438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e4179-9918-4ce3-b93d-b3761de20b44 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.337697 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f05895-e6d6-40c2-a97b-0ae0dfe2356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.338352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f05895-e6d6-40c2-a97b-0ae0dfe2356e admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.346311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f05895-e6d6-40c2-a97b-0ae0dfe2356e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.346311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f05895-e6d6-40c2-a97b-0ae0dfe2356e admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.346311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f05895-e6d6-40c2-a97b-0ae0dfe2356e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.346614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 803/1596] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.346717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f05895-e6d6-40c2-a97b-0ae0dfe2356e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.346845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e4179-9918-4ce3-b93d-b3761de20b44 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.356194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 794/1597] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/roles => generated 1427 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.356364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e4179-9918-4ce3-b93d-b3761de20b44 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.356466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f05895-e6d6-40c2-a97b-0ae0dfe2356e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.360408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df02deeb-8e5e-492b-b9c6-c1bccc4698e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.361156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02deeb-8e5e-492b-b9c6-c1bccc4698e1 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.361438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02deeb-8e5e-492b-b9c6-c1bccc4698e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.361728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df02deeb-8e5e-492b-b9c6-c1bccc4698e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.362369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02deeb-8e5e-492b-b9c6-c1bccc4698e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.363865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f05895-e6d6-40c2-a97b-0ae0dfe2356e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.364067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df02deeb-8e5e-492b-b9c6-c1bccc4698e1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.369333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fd8ba-84bc-4e0e-866d-fe3bfa8aca52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.369943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fd8ba-84bc-4e0e-866d-fe3bfa8aca52 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.370195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fd8ba-84bc-4e0e-866d-fe3bfa8aca52 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.370473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fd8ba-84bc-4e0e-866d-fe3bfa8aca52 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.371156 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755fd8ba-84bc-4e0e-866d-fe3bfa8aca52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.372732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755fd8ba-84bc-4e0e-866d-fe3bfa8aca52 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.376388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 804/1598] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.377692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc6e29-a052-4ba1-87db-d8c041e03c32 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.377982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc6e29-a052-4ba1-87db-d8c041e03c32 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.378280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc6e29-a052-4ba1-87db-d8c041e03c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.383096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 795/1599] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.384662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1ac7c-820c-4c1d-99eb-86dc39037785 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.384947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1ac7c-820c-4c1d-99eb-86dc39037785 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.385222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1ac7c-820c-4c1d-99eb-86dc39037785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.404903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-00f1ac7c-820c-4c1d-99eb-86dc39037785 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:17.406110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-00f1ac7c-820c-4c1d-99eb-86dc39037785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:17.413877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 796/1600] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:17.416450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9991933-ca04-4953-9c1e-8d38af3a8f1a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.416745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9991933-ca04-4953-9c1e-8d38af3a8f1a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.417459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9991933-ca04-4953-9c1e-8d38af3a8f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.422163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-fdfc6e29-a052-4ba1-87db-d8c041e03c32 None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:17.423751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fdfc6e29-a052-4ba1-87db-d8c041e03c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:17.445820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c9991933-ca04-4953-9c1e-8d38af3a8f1a None None] MFA Rules not processed for user `c13efb9d7b98426dba25a64c9c15b6e8`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:17.447049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c9991933-ca04-4953-9c1e-8d38af3a8f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:17.494732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 805/1601] 10.222.0.105 () {58 vars in 1064 bytes} [Wed Aug 24 04:59:17 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:17.495609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc6e29-a052-4ba1-87db-d8c041e03c32 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.506030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc6e29-a052-4ba1-87db-d8c041e03c32 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.510850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2976-1bf8-4a46-ac65-fdcec5429896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.511528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2976-1bf8-4a46-ac65-fdcec5429896 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.511810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2976-1bf8-4a46-ac65-fdcec5429896 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.512083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2976-1bf8-4a46-ac65-fdcec5429896 admin admin] PATH_INFO: `/v3/projects/d373da7992054abfae0564708529a973/users/842e7dea2bb24389b900d29351a0bd33/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.514869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 797/1602] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:17.515712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9991933-ca04-4953-9c1e-8d38af3a8f1a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.525932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9991933-ca04-4953-9c1e-8d38af3a8f1a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.527735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2976-1bf8-4a46-ac65-fdcec5429896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d373da7992054abfae0564708529a973, user_id=842e7dea2bb24389b900d29351a0bd33, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.530396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2976-1bf8-4a46-ac65-fdcec5429896 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.530522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1222a4-f9ae-47a8-a358-6be87cc8abdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.531114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1222a4-f9ae-47a8-a358-6be87cc8abdc admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.531388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1222a4-f9ae-47a8-a358-6be87cc8abdc admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.531660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1222a4-f9ae-47a8-a358-6be87cc8abdc admin admin] PATH_INFO: `/v3/projects/a26395fa888743908b6b32261d80dc43/users/2d9f1ce89ead453abf5a0b431e07f888/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.547038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1222a4-f9ae-47a8-a358-6be87cc8abdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26395fa888743908b6b32261d80dc43, user_id=2d9f1ce89ead453abf5a0b431e07f888, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.549734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1222a4-f9ae-47a8-a358-6be87cc8abdc admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.608746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 806/1603] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:17 2022] PUT /identity/v3/projects/d373da7992054abfae0564708529a973/users/842e7dea2bb24389b900d29351a0bd33/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:17.609497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2976-1bf8-4a46-ac65-fdcec5429896 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.619823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:17.666029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 798/1604] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:17 2022] PUT /identity/v3/projects/a26395fa888743908b6b32261d80dc43/users/2d9f1ce89ead453abf5a0b431e07f888/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:17.672585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2976-1bf8-4a46-ac65-fdcec5429896 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.677500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1222a4-f9ae-47a8-a358-6be87cc8abdc admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.731009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371df5d8-ef5d-4057-a9c3-7190eeeac47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.731741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371df5d8-ef5d-4057-a9c3-7190eeeac47b admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.732143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371df5d8-ef5d-4057-a9c3-7190eeeac47b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.732490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371df5d8-ef5d-4057-a9c3-7190eeeac47b admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.733247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371df5d8-ef5d-4057-a9c3-7190eeeac47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.735003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371df5d8-ef5d-4057-a9c3-7190eeeac47b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.739945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1222a4-f9ae-47a8-a358-6be87cc8abdc admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.744557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0578bb1-82e7-4de4-8f64-565ed5a7af65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.745210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0578bb1-82e7-4de4-8f64-565ed5a7af65 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.745482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0578bb1-82e7-4de4-8f64-565ed5a7af65 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.745749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0578bb1-82e7-4de4-8f64-565ed5a7af65 admin admin] PATH_INFO: `/v3/projects/bd6824ae6edd43fcbddb58581134098c/users/b11da4dac5804252b7310be096ae5d53/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.746122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 807/1605] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/roles => generated 1427 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:17.746984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371df5d8-ef5d-4057-a9c3-7190eeeac47b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.756950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371df5d8-ef5d-4057-a9c3-7190eeeac47b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.760881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0578bb1-82e7-4de4-8f64-565ed5a7af65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6824ae6edd43fcbddb58581134098c, user_id=b11da4dac5804252b7310be096ae5d53, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.762305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e90c2-3d1e-4525-a782-4b58b47ce205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.762970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e90c2-3d1e-4525-a782-4b58b47ce205 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.763283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e90c2-3d1e-4525-a782-4b58b47ce205 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.763592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e90c2-3d1e-4525-a782-4b58b47ce205 admin admin] PATH_INFO: `/v3/projects/f2fd5d378aba473ab58753aecb74a065/users/51926dd0d6934aed907806a5cb14dc2f/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.763702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0578bb1-82e7-4de4-8f64-565ed5a7af65 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.778958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e90c2-3d1e-4525-a782-4b58b47ce205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fd5d378aba473ab58753aecb74a065, user_id=51926dd0d6934aed907806a5cb14dc2f, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:17.781737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e90c2-3d1e-4525-a782-4b58b47ce205 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:17.858728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:17.864607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 799/1606] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:17 2022] PUT /identity/v3/projects/bd6824ae6edd43fcbddb58581134098c/users/b11da4dac5804252b7310be096ae5d53/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:17.865426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0578bb1-82e7-4de4-8f64-565ed5a7af65 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.866992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c0578bb1-82e7-4de4-8f64-565ed5a7af65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:17.894031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 808/1607] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:17 2022] PUT /identity/v3/projects/f2fd5d378aba473ab58753aecb74a065/users/51926dd0d6934aed907806a5cb14dc2f/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:17.899010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e90c2-3d1e-4525-a782-4b58b47ce205 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:17.925652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0578bb1-82e7-4de4-8f64-565ed5a7af65 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.970625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e90c2-3d1e-4525-a782-4b58b47ce205 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:17.975286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa595718-9870-49a3-8078-aed111953e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.975896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa595718-9870-49a3-8078-aed111953e90 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.976118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71336436-9ea3-466e-be40-db294a19df0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:17.976237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa595718-9870-49a3-8078-aed111953e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.976430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa595718-9870-49a3-8078-aed111953e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:17.977434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71336436-9ea3-466e-be40-db294a19df0b service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:17.977940 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71336436-9ea3-466e-be40-db294a19df0b service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:17.978317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71336436-9ea3-466e-be40-db294a19df0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.003049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:18.036628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71336436-9ea3-466e-be40-db294a19df0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.038197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71336436-9ea3-466e-be40-db294a19df0b service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.038548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa595718-9870-49a3-8078-aed111953e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.040668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa595718-9870-49a3-8078-aed111953e90 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.049193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 809/1608] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:18.049926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71336436-9ea3-466e-be40-db294a19df0b service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.050441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 800/1609] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:18.056920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c79c7-4f17-4aaa-9aa6-8d04e40d92fe None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.056920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c79c7-4f17-4aaa-9aa6-8d04e40d92fe None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.056920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166c79c7-4f17-4aaa-9aa6-8d04e40d92fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.061662 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71336436-9ea3-466e-be40-db294a19df0b service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:18.066968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252150a-4dad-42ce-985b-5cee7b274a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:18.067721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252150a-4dad-42ce-985b-5cee7b274a28 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.068092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252150a-4dad-42ce-985b-5cee7b274a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.068623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252150a-4dad-42ce-985b-5cee7b274a28 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.069349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252150a-4dad-42ce-985b-5cee7b274a28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.071894 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252150a-4dad-42ce-985b-5cee7b274a28 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.073712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-166c79c7-4f17-4aaa-9aa6-8d04e40d92fe None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:18.075473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-166c79c7-4f17-4aaa-9aa6-8d04e40d92fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:18.083331 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 801/1610] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:18.084526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab8abe-dad2-4290-bfec-d183fd836152 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.084935 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab8abe-dad2-4290-bfec-d183fd836152 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.085213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab8abe-dad2-4290-bfec-d183fd836152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.114212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-fcab8abe-dad2-4290-bfec-d183fd836152 None None] MFA Rules not processed for user `2d9f1ce89ead453abf5a0b431e07f888`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:18.119914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-fcab8abe-dad2-4290-bfec-d183fd836152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:18.162271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:18.186613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 810/1611] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:18 2022] POST /identity/v3/projects => generated 380 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:18.189695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252150a-4dad-42ce-985b-5cee7b274a28 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.192065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 802/1612] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:18.193445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6902b-fafe-45f1-b000-8641f3d10a9d None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.193695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6902b-fafe-45f1-b000-8641f3d10a9d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.194107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6902b-fafe-45f1-b000-8641f3d10a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.221818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-65f6902b-fafe-45f1-b000-8641f3d10a9d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:18.223190 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-65f6902b-fafe-45f1-b000-8641f3d10a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:18.280941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252150a-4dad-42ce-985b-5cee7b274a28 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:18.286291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4821812-d805-46c5-a272-7843c22a2ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:18.287055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4821812-d805-46c5-a272-7843c22a2ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.287337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4821812-d805-46c5-a272-7843c22a2ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.287607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4821812-d805-46c5-a272-7843c22a2ccf admin admin] PATH_INFO: `/v3/projects/fad6d7352bc144929743ca7373a41800/users/e45111c49aa74f76874d3a0c747034ba/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.300240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 803/1613] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:18.301662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b0211-7b50-4a4f-b27c-70105c1ac132 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.301956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b0211-7b50-4a4f-b27c-70105c1ac132 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.302232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b0211-7b50-4a4f-b27c-70105c1ac132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.312545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4821812-d805-46c5-a272-7843c22a2ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad6d7352bc144929743ca7373a41800, user_id=e45111c49aa74f76874d3a0c747034ba, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.315251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4821812-d805-46c5-a272-7843c22a2ccf admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.322361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-a91b0211-7b50-4a4f-b27c-70105c1ac132 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:18.326581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a91b0211-7b50-4a4f-b27c-70105c1ac132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:18.331754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 804/1614] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:18.332565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b0211-7b50-4a4f-b27c-70105c1ac132 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.386119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 811/1615] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:18 2022] PUT /identity/v3/projects/fad6d7352bc144929743ca7373a41800/users/e45111c49aa74f76874d3a0c747034ba/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:18.386119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4821812-d805-46c5-a272-7843c22a2ccf admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.393395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b0211-7b50-4a4f-b27c-70105c1ac132 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:18.462874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4821812-d805-46c5-a272-7843c22a2ccf admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:18.467509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d33599-0a91-4549-8a0a-7680dc9b902e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:18.468117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19078ced-3570-4f9a-9fd2-29e22806053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:18.468268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d33599-0a91-4549-8a0a-7680dc9b902e service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.468484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d33599-0a91-4549-8a0a-7680dc9b902e service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.468765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d33599-0a91-4549-8a0a-7680dc9b902e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.469101 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19078ced-3570-4f9a-9fd2-29e22806053c admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.469457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19078ced-3570-4f9a-9fd2-29e22806053c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.469730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19078ced-3570-4f9a-9fd2-29e22806053c admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.470361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19078ced-3570-4f9a-9fd2-29e22806053c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.472305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19078ced-3570-4f9a-9fd2-29e22806053c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.479546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d33599-0a91-4549-8a0a-7680dc9b902e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.482025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d33599-0a91-4549-8a0a-7680dc9b902e service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.491882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 805/1616] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:18.492710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d33599-0a91-4549-8a0a-7680dc9b902e service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.502740 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d33599-0a91-4549-8a0a-7680dc9b902e service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:18.507471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4bb45d-2db6-4d11-a0c8-cab6af12b1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:18.514834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bb45d-2db6-4d11-a0c8-cab6af12b1b1 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.514834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bb45d-2db6-4d11-a0c8-cab6af12b1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.514834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4bb45d-2db6-4d11-a0c8-cab6af12b1b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.514834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bb45d-2db6-4d11-a0c8-cab6af12b1b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.514834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4bb45d-2db6-4d11-a0c8-cab6af12b1b1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.576036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 812/1617] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:18 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:18.576036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c973d-8f33-4b02-969c-86c500888c16 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.576036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c973d-8f33-4b02-969c-86c500888c16 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.576036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c973d-8f33-4b02-969c-86c500888c16 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.576531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 813/1618] 10.222.0.105 () {54 vars in 917 bytes} [Wed Aug 24 04:59:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:59:18.579181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c973d-8f33-4b02-969c-86c500888c16 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.593748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c973d-8f33-4b02-969c-86c500888c16 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:18.597280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434853e-c103-45a8-a5ba-89cb2771db15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:18.597862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434853e-c103-45a8-a5ba-89cb2771db15 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.598136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434853e-c103-45a8-a5ba-89cb2771db15 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.598402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434853e-c103-45a8-a5ba-89cb2771db15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.614841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434853e-c103-45a8-a5ba-89cb2771db15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.614841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9434853e-c103-45a8-a5ba-89cb2771db15 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.620953 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 814/1619] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:18.621839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434853e-c103-45a8-a5ba-89cb2771db15 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.625486 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:18.629832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9434853e-c103-45a8-a5ba-89cb2771db15 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:18.633731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beb485-05bc-4e27-9ab5-2426f12c7013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:18.637298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb485-05bc-4e27-9ab5-2426f12c7013 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.637298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb485-05bc-4e27-9ab5-2426f12c7013 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.637298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1beb485-05bc-4e27-9ab5-2426f12c7013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.644702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1beb485-05bc-4e27-9ab5-2426f12c7013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.646136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1beb485-05bc-4e27-9ab5-2426f12c7013 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.660639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 806/1620] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:18 2022] POST /identity/v3/projects => generated 372 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:18.660639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814173e3-9a86-4af3-9f31-c04971fd6ff6 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.660639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814173e3-9a86-4af3-9f31-c04971fd6ff6 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.660639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814173e3-9a86-4af3-9f31-c04971fd6ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.685415 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 815/1621] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:18.687877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beb485-05bc-4e27-9ab5-2426f12c7013 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.697298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-814173e3-9a86-4af3-9f31-c04971fd6ff6 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:18.698873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-814173e3-9a86-4af3-9f31-c04971fd6ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:18.708477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1beb485-05bc-4e27-9ab5-2426f12c7013 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:18.722753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:18.781135 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 807/1622] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:18.782382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c821c-1253-440f-9802-9f965b41578c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.782666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c821c-1253-440f-9802-9f965b41578c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.782945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c821c-1253-440f-9802-9f965b41578c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.783310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5413a-21d7-4bff-9b55-a60493481565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:18.787153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5413a-21d7-4bff-9b55-a60493481565 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.787921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5413a-21d7-4bff-9b55-a60493481565 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.789042 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5413a-21d7-4bff-9b55-a60493481565 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.790841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5413a-21d7-4bff-9b55-a60493481565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.793900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5413a-21d7-4bff-9b55-a60493481565 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.803635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-dd5c821c-1253-440f-9802-9f965b41578c None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:18.805033 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-dd5c821c-1253-440f-9802-9f965b41578c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:18.807682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 816/1623] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:18 2022] GET /identity/v3/roles => generated 1427 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:18.808630 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5413a-21d7-4bff-9b55-a60493481565 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.820048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5413a-21d7-4bff-9b55-a60493481565 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:18.826890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6fa56-da10-489b-8ed0-9e83331090f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:18.829134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6fa56-da10-489b-8ed0-9e83331090f5 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.829134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6fa56-da10-489b-8ed0-9e83331090f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.829494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6fa56-da10-489b-8ed0-9e83331090f5 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.830588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6fa56-da10-489b-8ed0-9e83331090f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.832921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a6fa56-da10-489b-8ed0-9e83331090f5 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.889032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 808/1624] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 04:59:18 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:18.892310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c821c-1253-440f-9802-9f965b41578c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.965383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c821c-1253-440f-9802-9f965b41578c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:18.970116 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108af2e3-2676-4701-8b92-7ea28b22b1db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:18.971049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108af2e3-2676-4701-8b92-7ea28b22b1db service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:18.971381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108af2e3-2676-4701-8b92-7ea28b22b1db service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:18.973636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108af2e3-2676-4701-8b92-7ea28b22b1db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:18.983393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 817/1625] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:18 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:18.985066 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6fa56-da10-489b-8ed0-9e83331090f5 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:18.989279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108af2e3-2676-4701-8b92-7ea28b22b1db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:18.990813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108af2e3-2676-4701-8b92-7ea28b22b1db service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:18.999758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6fa56-da10-489b-8ed0-9e83331090f5 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:19.003000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 809/1626] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:19.007200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108af2e3-2676-4701-8b92-7ea28b22b1db service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:19.007338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f79ab4-355d-466f-a726-0a4e99e311f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:19.008734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f79ab4-355d-466f-a726-0a4e99e311f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.009209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f79ab4-355d-466f-a726-0a4e99e311f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.009979 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f79ab4-355d-466f-a726-0a4e99e311f6 admin admin] PATH_INFO: `/v3/projects/4caaae6efb3f421e923b8050fddcd15a/users/5d7dc14814784102978d2930604c956f/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.036963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f79ab4-355d-466f-a726-0a4e99e311f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4caaae6efb3f421e923b8050fddcd15a, user_id=5d7dc14814784102978d2930604c956f, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:19.041181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f79ab4-355d-466f-a726-0a4e99e311f6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:19.083582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108af2e3-2676-4701-8b92-7ea28b22b1db service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:19.088255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727af64-b96f-407d-b7b2-357ad90bbedf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:19.088860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727af64-b96f-407d-b7b2-357ad90bbedf service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.089135 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727af64-b96f-407d-b7b2-357ad90bbedf service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.089408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7727af64-b96f-407d-b7b2-357ad90bbedf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.098781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:19.101195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727af64-b96f-407d-b7b2-357ad90bbedf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:19.102762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7727af64-b96f-407d-b7b2-357ad90bbedf service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:19.114244 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 810/1627] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:19 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:19.115304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727af64-b96f-407d-b7b2-357ad90bbedf service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:19.138154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7727af64-b96f-407d-b7b2-357ad90bbedf service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:19.141250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 818/1628] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:18 2022] PUT /identity/v3/projects/4caaae6efb3f421e923b8050fddcd15a/users/5d7dc14814784102978d2930604c956f/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:19.145094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f79ab4-355d-466f-a726-0a4e99e311f6 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:19.235379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d45adc-e9ec-40a2-9739-c752c33f0f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:19.236009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d45adc-e9ec-40a2-9739-c752c33f0f3f admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.236434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d45adc-e9ec-40a2-9739-c752c33f0f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.236707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d45adc-e9ec-40a2-9739-c752c33f0f3f admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.237470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d45adc-e9ec-40a2-9739-c752c33f0f3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:19.240898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d45adc-e9ec-40a2-9739-c752c33f0f3f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:19.258118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 811/1629] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:19 2022] GET /identity/v3/roles => generated 1427 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:19.262146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d45adc-e9ec-40a2-9739-c752c33f0f3f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:19.274520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d45adc-e9ec-40a2-9739-c752c33f0f3f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:19.280508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361354f-f19a-4edd-a9c1-92ed7057b6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:19.281352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361354f-f19a-4edd-a9c1-92ed7057b6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.281733 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361354f-f19a-4edd-a9c1-92ed7057b6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.282175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361354f-f19a-4edd-a9c1-92ed7057b6fa admin admin] PATH_INFO: `/v3/projects/dcc679824bc34f4dafbd03891462278a/users/a6a942b6d0184ca190304fe847224a0a/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.283599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f79ab4-355d-466f-a726-0a4e99e311f6 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:19.298200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a169667-b5cd-41c3-b0d0-db07a91cb3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:19.299173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a169667-b5cd-41c3-b0d0-db07a91cb3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.299452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a169667-b5cd-41c3-b0d0-db07a91cb3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.299728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a169667-b5cd-41c3-b0d0-db07a91cb3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.300409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a169667-b5cd-41c3-b0d0-db07a91cb3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:19.301889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361354f-f19a-4edd-a9c1-92ed7057b6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc679824bc34f4dafbd03891462278a, user_id=a6a942b6d0184ca190304fe847224a0a, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:19.306668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a169667-b5cd-41c3-b0d0-db07a91cb3e5 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:19.320833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361354f-f19a-4edd-a9c1-92ed7057b6fa admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:19.333183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 819/1630] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:19 2022] GET /identity/v3/roles => generated 1427 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:19.336843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a169667-b5cd-41c3-b0d0-db07a91cb3e5 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:19.353827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a169667-b5cd-41c3-b0d0-db07a91cb3e5 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:19.370065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0b22a-5f07-4522-8290-3f8811b4aaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:19.370893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b22a-5f07-4522-8290-3f8811b4aaa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.371274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b22a-5f07-4522-8290-3f8811b4aaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.371556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0b22a-5f07-4522-8290-3f8811b4aaa2 admin admin] PATH_INFO: `/v3/projects/4caaae6efb3f421e923b8050fddcd15a/users/5d7dc14814784102978d2930604c956f/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.374082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0b22a-5f07-4522-8290-3f8811b4aaa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4caaae6efb3f421e923b8050fddcd15a, user_id=5d7dc14814784102978d2930604c956f, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:19.377999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0b22a-5f07-4522-8290-3f8811b4aaa2 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:19.395512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 820/1631] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:19 2022] PUT /identity/v3/projects/4caaae6efb3f421e923b8050fddcd15a/users/5d7dc14814784102978d2930604c956f/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:19.406589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89eb82-0019-44ef-b428-024b12bff598 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.407058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89eb82-0019-44ef-b428-024b12bff598 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.407596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89eb82-0019-44ef-b428-024b12bff598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.436774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 812/1632] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:19 2022] PUT /identity/v3/projects/dcc679824bc34f4dafbd03891462278a/users/a6a942b6d0184ca190304fe847224a0a/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:19.438655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361354f-f19a-4edd-a9c1-92ed7057b6fa admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:19.472404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-6e89eb82-0019-44ef-b428-024b12bff598 None None] MFA Rules not processed for user `5d7dc14814784102978d2930604c956f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:19.474322 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6e89eb82-0019-44ef-b428-024b12bff598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:19.494335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:19.594573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361354f-f19a-4edd-a9c1-92ed7057b6fa admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:19.600397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a0702-205c-4415-92a9-efc9b60916da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:19.612901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a0702-205c-4415-92a9-efc9b60916da admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.612901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a0702-205c-4415-92a9-efc9b60916da admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.612901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a0702-205c-4415-92a9-efc9b60916da admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.612901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a0702-205c-4415-92a9-efc9b60916da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:19.612901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a0702-205c-4415-92a9-efc9b60916da admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:19.623342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 813/1633] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:19 2022] GET /identity/v3/roles => generated 1427 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:19.623342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a0702-205c-4415-92a9-efc9b60916da admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:19.623993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 821/1634] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:19 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:59:19.634383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89eb82-0019-44ef-b428-024b12bff598 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:19.642673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a0702-205c-4415-92a9-efc9b60916da admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:19.657538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727be93-6ac9-483e-9ede-8fb545eef1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:19.659246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727be93-6ac9-483e-9ede-8fb545eef1a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.659246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727be93-6ac9-483e-9ede-8fb545eef1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.659246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727be93-6ac9-483e-9ede-8fb545eef1a0 admin admin] PATH_INFO: `/v3/projects/dcc679824bc34f4dafbd03891462278a/users/a6a942b6d0184ca190304fe847224a0a/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.668632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727be93-6ac9-483e-9ede-8fb545eef1a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc679824bc34f4dafbd03891462278a, user_id=a6a942b6d0184ca190304fe847224a0a, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:19.671475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a727be93-6ac9-483e-9ede-8fb545eef1a0 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:19.693135 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 814/1635] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:19 2022] PUT /identity/v3/projects/dcc679824bc34f4dafbd03891462278a/users/a6a942b6d0184ca190304fe847224a0a/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:19.704089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033562d-a7b6-4920-ad9b-9c6fc4f60479 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.704365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033562d-a7b6-4920-ad9b-9c6fc4f60479 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.704709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033562d-a7b6-4920-ad9b-9c6fc4f60479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.727548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89eb82-0019-44ef-b428-024b12bff598 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:19.738647 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5033562d-a7b6-4920-ad9b-9c6fc4f60479 None None] MFA Rules not processed for user `a6a942b6d0184ca190304fe847224a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:19.740380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5033562d-a7b6-4920-ad9b-9c6fc4f60479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:19.785161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90feb22-b735-4ae5-ad4b-378d5340a117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:19.786706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90feb22-b735-4ae5-ad4b-378d5340a117 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:19.787043 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90feb22-b735-4ae5-ad4b-378d5340a117 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:19.787750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90feb22-b735-4ae5-ad4b-378d5340a117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:19.837764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:19.918621 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 815/1636] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:59:19.937038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90feb22-b735-4ae5-ad4b-378d5340a117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:19.938808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90feb22-b735-4ae5-ad4b-378d5340a117 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:19.964765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 822/1637] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:19 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:59:20.076111 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033562d-a7b6-4920-ad9b-9c6fc4f60479 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:20.094481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033562d-a7b6-4920-ad9b-9c6fc4f60479 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:20.100407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441c8b4-4056-4e2c-9cb6-e53d2c4eeab9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:20.102555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441c8b4-4056-4e2c-9cb6-e53d2c4eeab9 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:20.102828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441c8b4-4056-4e2c-9cb6-e53d2c4eeab9 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:20.105416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c441c8b4-4056-4e2c-9cb6-e53d2c4eeab9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:20.117468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441c8b4-4056-4e2c-9cb6-e53d2c4eeab9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:20.119602 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c441c8b4-4056-4e2c-9cb6-e53d2c4eeab9 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:20.154889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 816/1638] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:20.516622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90feb22-b735-4ae5-ad4b-378d5340a117 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:20.540641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90feb22-b735-4ae5-ad4b-378d5340a117 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:20.546357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0110e-2ab2-4559-80e8-459d7c225ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:20.547143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0110e-2ab2-4559-80e8-459d7c225ccd service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:20.565021 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0110e-2ab2-4559-80e8-459d7c225ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:20.565747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0110e-2ab2-4559-80e8-459d7c225ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:20.592523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:20.669699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0110e-2ab2-4559-80e8-459d7c225ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:20.669699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd0110e-2ab2-4559-80e8-459d7c225ccd service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:20.670332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 823/1639] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:20.745862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441c8b4-4056-4e2c-9cb6-e53d2c4eeab9 service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:20.763888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c441c8b4-4056-4e2c-9cb6-e53d2c4eeab9 service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:20.777701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada82db-b52f-4032-8b4e-da9338f48526 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:20.778334 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada82db-b52f-4032-8b4e-da9338f48526 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:20.779754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada82db-b52f-4032-8b4e-da9338f48526 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:20.780029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada82db-b52f-4032-8b4e-da9338f48526 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:20.908292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada82db-b52f-4032-8b4e-da9338f48526 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:20.909643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada82db-b52f-4032-8b4e-da9338f48526 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:20.927863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 817/1640] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:21.122629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37162d-4700-4bfe-bd40-d72e95b946d9 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:21.124010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37162d-4700-4bfe-bd40-d72e95b946d9 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:21.124861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37162d-4700-4bfe-bd40-d72e95b946d9 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:21.128863 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 824/1641] 10.222.0.105 () {54 vars in 967 bytes} [Wed Aug 24 04:59:21 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:59:21.137856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e28bec-e9a9-4b68-a001-69d42c74a60d None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:21.139755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e28bec-e9a9-4b68-a001-69d42c74a60d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:21.139755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e28bec-e9a9-4b68-a001-69d42c74a60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:21.162541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-57e28bec-e9a9-4b68-a001-69d42c74a60d None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:21.167561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-57e28bec-e9a9-4b68-a001-69d42c74a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:21.319583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 818/1642] 10.222.0.105 () {58 vars in 1064 bytes} [Wed Aug 24 04:59:21 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:21.326334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95681e66-6ae5-4804-a839-ab09587f2f7f None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:21.326800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95681e66-6ae5-4804-a839-ab09587f2f7f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:21.327191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95681e66-6ae5-4804-a839-ab09587f2f7f None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:21.329303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 825/1643] 10.222.0.105 () {54 vars in 917 bytes} [Wed Aug 24 04:59:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:59:21.335154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c8b0f-996d-4678-a96e-cee5044652f9 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:21.335632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c8b0f-996d-4678-a96e-cee5044652f9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:21.336032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853c8b0f-996d-4678-a96e-cee5044652f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:21.368191 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-853c8b0f-996d-4678-a96e-cee5044652f9 None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:21.369944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-853c8b0f-996d-4678-a96e-cee5044652f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:21.387382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 819/1644] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 04:59:21 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:21.400823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95681e66-6ae5-4804-a839-ab09587f2f7f None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:21.415202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95681e66-6ae5-4804-a839-ab09587f2f7f None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:21.421775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608d40-0c7d-4729-bb33-7b3e613e083b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:21.423166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608d40-0c7d-4729-bb33-7b3e613e083b service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:21.423655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608d40-0c7d-4729-bb33-7b3e613e083b service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:21.424212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3608d40-0c7d-4729-bb33-7b3e613e083b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:21.438503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3608d40-0c7d-4729-bb33-7b3e613e083b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:21.449613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3608d40-0c7d-4729-bb33-7b3e613e083b service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:21.467386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 826/1645] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:21 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:30.027318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca846c-a2b4-410b-87a4-63f646ef04c4 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:30.027907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca846c-a2b4-410b-87a4-63f646ef04c4 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:30.027907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca846c-a2b4-410b-87a4-63f646ef04c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:30.113914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-11ca846c-a2b4-410b-87a4-63f646ef04c4 None None] MFA Rules not processed for user `e45111c49aa74f76874d3a0c747034ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:30.118928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-11ca846c-a2b4-410b-87a4-63f646ef04c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:30.260865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:30.359831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 820/1646] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:30.392529 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608d40-0c7d-4729-bb33-7b3e613e083b service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:30.412170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3608d40-0c7d-4729-bb33-7b3e613e083b service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:30.451611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ee1e1-be52-49eb-937a-19f2b3cbc575 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:30.452454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ee1e1-be52-49eb-937a-19f2b3cbc575 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:30.452874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ee1e1-be52-49eb-937a-19f2b3cbc575 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:30.453286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ee1e1-be52-49eb-937a-19f2b3cbc575 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:30.472125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ee1e1-be52-49eb-937a-19f2b3cbc575 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:30.474576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ee1e1-be52-49eb-937a-19f2b3cbc575 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:30.512953 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 827/1647] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:30 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:33.179065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda7090-5f84-4796-ba6d-fb3ad12e7dac None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:33.179640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda7090-5f84-4796-ba6d-fb3ad12e7dac None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:33.179725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda7090-5f84-4796-ba6d-fb3ad12e7dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:33.264933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-aeda7090-5f84-4796-ba6d-fb3ad12e7dac None None] MFA Rules not processed for user `b11da4dac5804252b7310be096ae5d53`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:33.266170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-aeda7090-5f84-4796-ba6d-fb3ad12e7dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:33.407242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 821/1648] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:33.423727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ee1e1-be52-49eb-937a-19f2b3cbc575 service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:33.445804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ee1e1-be52-49eb-937a-19f2b3cbc575 service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:33.460578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053cffdb-b6b3-4a18-b471-813de7789e6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:33.461192 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053cffdb-b6b3-4a18-b471-813de7789e6a service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:33.461860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053cffdb-b6b3-4a18-b471-813de7789e6a service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:33.462537 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053cffdb-b6b3-4a18-b471-813de7789e6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:33.480300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053cffdb-b6b3-4a18-b471-813de7789e6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:33.486383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053cffdb-b6b3-4a18-b471-813de7789e6a service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:33.536994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 828/1649] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:35.248211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a5786-e304-4f1d-895f-f71a09dcee0c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:35.250412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a5786-e304-4f1d-895f-f71a09dcee0c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:35.250874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a5786-e304-4f1d-895f-f71a09dcee0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:35.401030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ec7a5786-e304-4f1d-895f-f71a09dcee0c None None] MFA Rules not processed for user `842e7dea2bb24389b900d29351a0bd33`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:35.417208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ec7a5786-e304-4f1d-895f-f71a09dcee0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:35.629606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 822/1650] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:35 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:35.644613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d01dd6-20d2-4152-8474-95fbab4bf4a3 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:35.645453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d01dd6-20d2-4152-8474-95fbab4bf4a3 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:35.657191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d01dd6-20d2-4152-8474-95fbab4bf4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:35.677207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a5786-e304-4f1d-895f-f71a09dcee0c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:35.742892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a5786-e304-4f1d-895f-f71a09dcee0c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:35.748507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ca6be-f3c0-453a-a239-96d3fd0d8f9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:35.749464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ca6be-f3c0-453a-a239-96d3fd0d8f9f service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:35.749950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ca6be-f3c0-453a-a239-96d3fd0d8f9f service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:35.750407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ca6be-f3c0-453a-a239-96d3fd0d8f9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:35.765000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ca6be-f3c0-453a-a239-96d3fd0d8f9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:35.773509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ca6be-f3c0-453a-a239-96d3fd0d8f9f service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:35.777199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-19d01dd6-20d2-4152-8474-95fbab4bf4a3 None None] MFA Rules not processed for user `312b0174466b4b00be19d78bdc01fdb4`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:35.779708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-19d01dd6-20d2-4152-8474-95fbab4bf4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:35.830388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 823/1651] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:35 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:35.996053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 829/1652] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:36.009219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db5ff9-8f55-4ee9-b954-c4eea8e71634 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:36.009886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db5ff9-8f55-4ee9-b954-c4eea8e71634 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:36.010303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28db5ff9-8f55-4ee9-b954-c4eea8e71634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:36.073704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-28db5ff9-8f55-4ee9-b954-c4eea8e71634 None None] MFA Rules not processed for user `312b0174466b4b00be19d78bdc01fdb4`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:36.079766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-28db5ff9-8f55-4ee9-b954-c4eea8e71634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:36.110249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 824/1653] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:36.250814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d01dd6-20d2-4152-8474-95fbab4bf4a3 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:36.268236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d01dd6-20d2-4152-8474-95fbab4bf4a3 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:36.287798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c1f9c-8727-4104-a851-c2fdfe7689be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:36.292015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c1f9c-8727-4104-a851-c2fdfe7689be service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:36.292398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c1f9c-8727-4104-a851-c2fdfe7689be service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:36.292779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c1f9c-8727-4104-a851-c2fdfe7689be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:36.317476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c1f9c-8727-4104-a851-c2fdfe7689be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:36.319458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c1f9c-8727-4104-a851-c2fdfe7689be service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:36.343293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 830/1654] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:36 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:37.268976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c86dd-0b77-484e-b6fd-2f1fdd3984c6 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:37.269499 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c86dd-0b77-484e-b6fd-2f1fdd3984c6 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:37.269499 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c86dd-0b77-484e-b6fd-2f1fdd3984c6 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:37.271359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 825/1655] 10.222.0.105 () {54 vars in 917 bytes} [Wed Aug 24 04:59:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:59:37.280702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d37bd9-d1f8-45d7-a2a5-1265b5e6dcb6 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:37.281602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d37bd9-d1f8-45d7-a2a5-1265b5e6dcb6 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:37.282020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d37bd9-d1f8-45d7-a2a5-1265b5e6dcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:37.408744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-a5d37bd9-d1f8-45d7-a2a5-1265b5e6dcb6 None None] MFA Rules not processed for user `44d2b4015ac74b8b8d0a674419f34cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:37.429720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a5d37bd9-d1f8-45d7-a2a5-1265b5e6dcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:37.538878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:37.587704 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 831/1656] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 04:59:37 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:37.644341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c86dd-0b77-484e-b6fd-2f1fdd3984c6 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:37.666024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c86dd-0b77-484e-b6fd-2f1fdd3984c6 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:37.673669 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f191174-7798-4747-bb4e-f638deb73856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:37.674673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f191174-7798-4747-bb4e-f638deb73856 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:37.675179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f191174-7798-4747-bb4e-f638deb73856 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:37.675653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f191174-7798-4747-bb4e-f638deb73856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:37.690532 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f191174-7798-4747-bb4e-f638deb73856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:37.692579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f191174-7798-4747-bb4e-f638deb73856 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:37.704155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 826/1657] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:37 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:37.822516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9ed9-fc80-43af-acd9-f995a1937652 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:37.822795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9ed9-fc80-43af-acd9-f995a1937652 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:37.823462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9ed9-fc80-43af-acd9-f995a1937652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:37.894924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-142c9ed9-fc80-43af-acd9-f995a1937652 None None] MFA Rules not processed for user `51926dd0d6934aed907806a5cb14dc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:37.897727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-142c9ed9-fc80-43af-acd9-f995a1937652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:37.981062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bad0f-23c2-4f30-a3b4-cf6e106d3c29 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:37.981454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bad0f-23c2-4f30-a3b4-cf6e106d3c29 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:37.981853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bad0f-23c2-4f30-a3b4-cf6e106d3c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:38.026866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ab6bad0f-23c2-4f30-a3b4-cf6e106d3c29 None None] MFA Rules not processed for user `e45111c49aa74f76874d3a0c747034ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:38.028136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ab6bad0f-23c2-4f30-a3b4-cf6e106d3c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:38.063243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 827/1658] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:37 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:59:38.148665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 832/1659] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:37 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:38.158020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a16ba6-6721-4c03-a1cb-1c45cb570a03 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:38.158783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a16ba6-6721-4c03-a1cb-1c45cb570a03 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:38.165831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a16ba6-6721-4c03-a1cb-1c45cb570a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:38.230154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-60a16ba6-6721-4c03-a1cb-1c45cb570a03 None None] MFA Rules not processed for user `51926dd0d6934aed907806a5cb14dc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:38.236434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-60a16ba6-6721-4c03-a1cb-1c45cb570a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:38.264782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 828/1660] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:38 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:38.541814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c9ed9-fc80-43af-acd9-f995a1937652 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:38.564164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c9ed9-fc80-43af-acd9-f995a1937652 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:38.575263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0659a25-2d87-4f59-8965-08c151d1a899 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:38.576393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0659a25-2d87-4f59-8965-08c151d1a899 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:38.577415 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0659a25-2d87-4f59-8965-08c151d1a899 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:38.577848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0659a25-2d87-4f59-8965-08c151d1a899 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:38.619686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0659a25-2d87-4f59-8965-08c151d1a899 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:38.623562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0659a25-2d87-4f59-8965-08c151d1a899 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:38.650274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 833/1661] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:38 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:38.671804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3487c0-b4c3-4c77-95cb-4665a3a40d8c None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:38.671804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3487c0-b4c3-4c77-95cb-4665a3a40d8c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:38.671804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3487c0-b4c3-4c77-95cb-4665a3a40d8c None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:38.671804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 829/1662] 10.222.0.105 () {54 vars in 966 bytes} [Wed Aug 24 04:59:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:59:38.685518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080050ae-519d-4be5-8e7f-ee5c2098bb43 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:38.685807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080050ae-519d-4be5-8e7f-ee5c2098bb43 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:38.686089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080050ae-519d-4be5-8e7f-ee5c2098bb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:38.712187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-080050ae-519d-4be5-8e7f-ee5c2098bb43 None None] MFA Rules not processed for user `ad8825c294534a7c88877b48b5f8f07f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:38.713618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-080050ae-519d-4be5-8e7f-ee5c2098bb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:38.815794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72cfc8-d302-4d87-b8e5-aa80e4fdd48b None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:38.816077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72cfc8-d302-4d87-b8e5-aa80e4fdd48b None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:38.816508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72cfc8-d302-4d87-b8e5-aa80e4fdd48b None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:38.819271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 830/1663] 10.222.0.105 () {54 vars in 917 bytes} [Wed Aug 24 04:59:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:59:38.838697 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab7947-906b-4d73-9944-fe80646c8a66 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:38.839182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab7947-906b-4d73-9944-fe80646c8a66 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:38.841776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab7947-906b-4d73-9944-fe80646c8a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:38.898183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-73ab7947-906b-4d73-9944-fe80646c8a66 None None] MFA Rules not processed for user `44d2b4015ac74b8b8d0a674419f34cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:38.899791 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-73ab7947-906b-4d73-9944-fe80646c8a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:38.915979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 831/1664] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 04:59:38 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 24 04:59:38.919774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 834/1665] 10.222.0.105 () {58 vars in 1063 bytes} [Wed Aug 24 04:59:38 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:38.929012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080050ae-519d-4be5-8e7f-ee5c2098bb43 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:38.941675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab7947-906b-4d73-9944-fe80646c8a66 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:38.946727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:38.954825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab7947-906b-4d73-9944-fe80646c8a66 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:38.966494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080050ae-519d-4be5-8e7f-ee5c2098bb43 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:38.977833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7e8b6-aedb-45a8-9942-ba25508fb47e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:38.978462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7e8b6-aedb-45a8-9942-ba25508fb47e service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:38.978741 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7e8b6-aedb-45a8-9942-ba25508fb47e service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:38.979211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7e8b6-aedb-45a8-9942-ba25508fb47e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:38.987760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f5e603-a959-4eed-bdc6-6595c56bc5e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:38.993056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5e603-a959-4eed-bdc6-6595c56bc5e5 service placement] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:38.993679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5e603-a959-4eed-bdc6-6595c56bc5e5 service placement] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:38.994176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f5e603-a959-4eed-bdc6-6595c56bc5e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:39.004173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7e8b6-aedb-45a8-9942-ba25508fb47e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:39.006579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7e8b6-aedb-45a8-9942-ba25508fb47e service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:39.015947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-22f5e603-a959-4eed-bdc6-6595c56bc5e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:39.030421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 832/1666] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:38 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:59:39.052984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5e603-a959-4eed-bdc6-6595c56bc5e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:39.055365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f5e603-a959-4eed-bdc6-6595c56bc5e5 service placement] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:39.082154 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 835/1667] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:38 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:59:39.428058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6637fc-c7ad-43f3-aeea-e7b8ccb53a19 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:39.428561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6637fc-c7ad-43f3-aeea-e7b8ccb53a19 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:39.428972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6637fc-c7ad-43f3-aeea-e7b8ccb53a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:39.498694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-3a6637fc-c7ad-43f3-aeea-e7b8ccb53a19 None None] MFA Rules not processed for user `b11da4dac5804252b7310be096ae5d53`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:39.500595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3a6637fc-c7ad-43f3-aeea-e7b8ccb53a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:39.525073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 833/1668] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:40.823436 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d9a46-e754-4292-bad7-ea1523d4bbab None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:40.824375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d9a46-e754-4292-bad7-ea1523d4bbab None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:40.824830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508d9a46-e754-4292-bad7-ea1523d4bbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:40.856785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-508d9a46-e754-4292-bad7-ea1523d4bbab None None] MFA Rules not processed for user `51926dd0d6934aed907806a5cb14dc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:40.859920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-508d9a46-e754-4292-bad7-ea1523d4bbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:40.885168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 836/1669] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:40 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:41.631851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff93ee-72fa-43bd-86da-93534d248496 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:41.632809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff93ee-72fa-43bd-86da-93534d248496 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:41.633249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aff93ee-72fa-43bd-86da-93534d248496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:41.684408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-0aff93ee-72fa-43bd-86da-93534d248496 None None] MFA Rules not processed for user `842e7dea2bb24389b900d29351a0bd33`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:41.685874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0aff93ee-72fa-43bd-86da-93534d248496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:41.699213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 834/1670] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:43.888426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d9a46-e754-4292-bad7-ea1523d4bbab None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:44.022294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:44.056748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508d9a46-e754-4292-bad7-ea1523d4bbab None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:44.065439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:44.066385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:44.067543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:44.067887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] PATH_INFO: `/v3/users/00e30c4bef674af481586297243901db` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:44.069190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e30c4bef674af481586297243901db)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:44.070442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: WARNING py.warnings [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:59:44.070442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: warnings.warn(msg) Aug 24 04:59:44.070442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: Aug 24 04:59:44.082949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:44.374729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e30c4bef674af481586297243901db'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:44.375602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e30c4bef674af481586297243901db'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:44.377431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e30c4bef674af481586297243901db'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:44.514967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e30c4bef674af481586297243901db'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:44.564746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e30c4bef674af481586297243901db'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:44.654124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 837/1671] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 04:59:43 2022] DELETE /identity/v3/users/00e30c4bef674af481586297243901db => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:44.655793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd87abc-cddc-4b45-86f4-7050701ed797 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:44.655793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd87abc-cddc-4b45-86f4-7050701ed797 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:44.655793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd87abc-cddc-4b45-86f4-7050701ed797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:44.678307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-edd87abc-cddc-4b45-86f4-7050701ed797 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:44.695114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-edd87abc-cddc-4b45-86f4-7050701ed797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:44.860296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 835/1672] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:44.872455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:44.874121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:45.005503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9d4a7-7ab5-4143-b3f0-980f0876568c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:45.013234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd383183-f866-4533-970a-0d27490b2b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:45.018304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd383183-f866-4533-970a-0d27490b2b6a service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:45.018304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd383183-f866-4533-970a-0d27490b2b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:45.018304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd383183-f866-4533-970a-0d27490b2b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:45.027310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd383183-f866-4533-970a-0d27490b2b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:45.029420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd383183-f866-4533-970a-0d27490b2b6a service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:45.056237 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 838/1673] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:45.357958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77cea8-7626-42d4-ae7f-52d032b29c0a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:45.358437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77cea8-7626-42d4-ae7f-52d032b29c0a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:45.358854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77cea8-7626-42d4-ae7f-52d032b29c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:45.494598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-4b77cea8-7626-42d4-ae7f-52d032b29c0a None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:45.497566 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4b77cea8-7626-42d4-ae7f-52d032b29c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:45.628649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 836/1674] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 04:59:45 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:59:46.446310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd383183-f866-4533-970a-0d27490b2b6a service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:46.448246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-dd383183-f866-4533-970a-0d27490b2b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:46.476205 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77cea8-7626-42d4-ae7f-52d032b29c0a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:46.477949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4b77cea8-7626-42d4-ae7f-52d032b29c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:46.507710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77cea8-7626-42d4-ae7f-52d032b29c0a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:46.515357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:46.516370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:46.516781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:46.517205 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] PATH_INFO: `/v3/users/c13efb9d7b98426dba25a64c9c15b6e8` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:46.519763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13efb9d7b98426dba25a64c9c15b6e8)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:46.526996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:46.674752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:46.725942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd383183-f866-4533-970a-0d27490b2b6a service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:46.735641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658a622-eaad-4c2d-a95e-26aff8f9b510 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:46.736541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658a622-eaad-4c2d-a95e-26aff8f9b510 service swift] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:46.737005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658a622-eaad-4c2d-a95e-26aff8f9b510 service swift] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:46.737531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658a622-eaad-4c2d-a95e-26aff8f9b510 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:46.768155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658a622-eaad-4c2d-a95e-26aff8f9b510 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:46.772435 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658a622-eaad-4c2d-a95e-26aff8f9b510 service swift] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:46.797098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 839/1675] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 04:59:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 04:59:46.874828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:46.884522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13efb9d7b98426dba25a64c9c15b6e8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:46.885686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13efb9d7b98426dba25a64c9c15b6e8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:46.887198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13efb9d7b98426dba25a64c9c15b6e8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:47.016606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13efb9d7b98426dba25a64c9c15b6e8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:47.049489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13efb9d7b98426dba25a64c9c15b6e8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:47.107625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 837/1676] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 04:59:46 2022] DELETE /identity/v3/users/c13efb9d7b98426dba25a64c9c15b6e8 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:59:47.112855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa816cb1-c1df-4e22-ab61-46259e33a57d None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:47.113145 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa816cb1-c1df-4e22-ab61-46259e33a57d None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:47.114082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa816cb1-c1df-4e22-ab61-46259e33a57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:47.174417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-fa816cb1-c1df-4e22-ab61-46259e33a57d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:47.181877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fa816cb1-c1df-4e22-ab61-46259e33a57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:47.399504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 840/1677] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:47.712307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:47.713858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:47.939436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2bd9e-e379-43b2-ba62-b10fb38f69be admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:47.953032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853f378-0c87-477f-a3af-b436afd85a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:47.953869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f378-0c87-477f-a3af-b436afd85a40 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:47.956001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f378-0c87-477f-a3af-b436afd85a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:47.956373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa816cb1-c1df-4e22-ab61-46259e33a57d None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:47.958509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fa816cb1-c1df-4e22-ab61-46259e33a57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:47.958807 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853f378-0c87-477f-a3af-b436afd85a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:47.991280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f378-0c87-477f-a3af-b436afd85a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:47.992990 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853f378-0c87-477f-a3af-b436afd85a40 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:47.999691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa816cb1-c1df-4e22-ab61-46259e33a57d None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:48.024946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:48.032014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:48.035012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:48.036403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] PATH_INFO: `/v3/projects/904b8da68db84c3ea1f8785c08ecdc58` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:48.043710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] RBAC: Authorizing `identity:delete_project(project_id=904b8da68db84c3ea1f8785c08ecdc58)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:48.045234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 838/1678] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:48.047754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:48.403559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '904b8da68db84c3ea1f8785c08ecdc58', 'request_id': 'req-c9f1f5da-b17a-47ff-a138-f525f7c5636f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:48.438454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7e9e8-9116-437e-b270-52230acf067c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:48.438454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7e9e8-9116-437e-b270-52230acf067c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:48.438454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7e9e8-9116-437e-b270-52230acf067c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:48.458912 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '904b8da68db84c3ea1f8785c08ecdc58', 'request_id': 'req-c9f1f5da-b17a-47ff-a138-f525f7c5636f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:48.492616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '904b8da68db84c3ea1f8785c08ecdc58', 'request_id': 'req-c9f1f5da-b17a-47ff-a138-f525f7c5636f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:48.506737 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c3c7e9e8-9116-437e-b270-52230acf067c None None] MFA Rules not processed for user `44d2b4015ac74b8b8d0a674419f34cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:48.510428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c3c7e9e8-9116-437e-b270-52230acf067c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:48.624692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] The token cache is being invalidate because project 904b8da68db84c3ea1f8785c08ecdc58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:59:48.627754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:48.627754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c9f1f5da-b17a-47ff-a138-f525f7c5636f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:48.628564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 841/1679] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 04:59:47 2022] DELETE /identity/v3/projects/904b8da68db84c3ea1f8785c08ecdc58 => generated 0 bytes in 673 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:59:48.642968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5d543-fe36-4817-bf49-371daea00df8 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:48.648830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5d543-fe36-4817-bf49-371daea00df8 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:48.650311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5d543-fe36-4817-bf49-371daea00df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:48.723056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 839/1680] 10.222.0.102 () {58 vars in 1020 bytes} [Wed Aug 24 04:59:48 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:48.744630 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-55b5d543-fe36-4817-bf49-371daea00df8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:48.760353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-55b5d543-fe36-4817-bf49-371daea00df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:48.872220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:48.892401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 842/1681] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:48.893599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b1817-1ac4-4e3e-bca7-fd0165700ae9 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:48.894083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b1817-1ac4-4e3e-bca7-fd0165700ae9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:48.894388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b1817-1ac4-4e3e-bca7-fd0165700ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:48.995040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9e5b1817-1ac4-4e3e-bca7-fd0165700ae9 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:48.998390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9e5b1817-1ac4-4e3e-bca7-fd0165700ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:49.029718 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 840/1682] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:59:49.034588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5d543-fe36-4817-bf49-371daea00df8 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:49.051536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5d543-fe36-4817-bf49-371daea00df8 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:49.059652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdef6b-bb40-4173-92e3-48b6d8f02410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:49.060862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdef6b-bb40-4173-92e3-48b6d8f02410 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:49.061216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdef6b-bb40-4173-92e3-48b6d8f02410 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:49.061633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdef6b-bb40-4173-92e3-48b6d8f02410 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:49.062373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdef6b-bb40-4173-92e3-48b6d8f02410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:59:49.062870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdef6b-bb40-4173-92e3-48b6d8f02410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:49.064830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfdef6b-bb40-4173-92e3-48b6d8f02410 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:49.072635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b1817-1ac4-4e3e-bca7-fd0165700ae9 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:49.074062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9e5b1817-1ac4-4e3e-bca7-fd0165700ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:49.087288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 843/1683] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:59:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:59:49.088523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdef6b-bb40-4173-92e3-48b6d8f02410 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:49.100934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b1817-1ac4-4e3e-bca7-fd0165700ae9 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:49.105597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdef6b-bb40-4173-92e3-48b6d8f02410 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:49.111857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb316e-d787-4689-9c2d-094698cea362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:49.112587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb316e-d787-4689-9c2d-094698cea362 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:49.112858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb316e-d787-4689-9c2d-094698cea362 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:49.113446 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb316e-d787-4689-9c2d-094698cea362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:49.113868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a3649-05d7-4090-b740-1012be4175b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:49.114762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a3649-05d7-4090-b740-1012be4175b8 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:49.115263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a3649-05d7-4090-b740-1012be4175b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:49.125556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb316e-d787-4689-9c2d-094698cea362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:49.127138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a3649-05d7-4090-b740-1012be4175b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:49.127347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb316e-d787-4689-9c2d-094698cea362 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:49.137593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a3649-05d7-4090-b740-1012be4175b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:49.139686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 841/1684] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:49 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:49.140387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a3649-05d7-4090-b740-1012be4175b8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:49.324960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 844/1685] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:49 2022] POST /identity/v3/projects => generated 370 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:49.329126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb316e-d787-4689-9c2d-094698cea362 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:49.351544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb316e-d787-4689-9c2d-094698cea362 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:49.541757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:49.562508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b78e28-6a82-4da3-9aa2-cd40c1afea33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:49.565138 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b78e28-6a82-4da3-9aa2-cd40c1afea33 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:49.565138 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b78e28-6a82-4da3-9aa2-cd40c1afea33 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:49.566965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b78e28-6a82-4da3-9aa2-cd40c1afea33 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:49.567329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b78e28-6a82-4da3-9aa2-cd40c1afea33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:49.571378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b78e28-6a82-4da3-9aa2-cd40c1afea33 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:49.781867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 842/1686] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:49 2022] POST /identity/v3/users => generated 286 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:49.786523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a3649-05d7-4090-b740-1012be4175b8 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:49.806984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a3649-05d7-4090-b740-1012be4175b8 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:49.842860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa90fc0-9f7f-47e7-8057-7f1af9187594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:49.844785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa90fc0-9f7f-47e7-8057-7f1af9187594 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:49.845928 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa90fc0-9f7f-47e7-8057-7f1af9187594 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:49.846365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa90fc0-9f7f-47e7-8057-7f1af9187594 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:49.847838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa90fc0-9f7f-47e7-8057-7f1af9187594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:49.856340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa90fc0-9f7f-47e7-8057-7f1af9187594 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:49.923874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 845/1687] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:49 2022] GET /identity/v3/roles => generated 1427 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:49.924074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b78e28-6a82-4da3-9aa2-cd40c1afea33 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:49.957617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b78e28-6a82-4da3-9aa2-cd40c1afea33 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:49.964507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc51070-591b-4bda-806a-42f3ff8d2a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:49.965699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc51070-591b-4bda-806a-42f3ff8d2a92 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:49.966189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc51070-591b-4bda-806a-42f3ff8d2a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:49.966656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc51070-591b-4bda-806a-42f3ff8d2a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:49.967521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc51070-591b-4bda-806a-42f3ff8d2a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:49.969549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc51070-591b-4bda-806a-42f3ff8d2a92 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:50.006985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa90fc0-9f7f-47e7-8057-7f1af9187594 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:50.019354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 843/1688] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:49 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:50.038331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa90fc0-9f7f-47e7-8057-7f1af9187594 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:50.064174 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650e155-3769-4c9a-8704-22e70b233590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:50.065057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e155-3769-4c9a-8704-22e70b233590 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:50.065497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e155-3769-4c9a-8704-22e70b233590 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:50.065953 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e155-3769-4c9a-8704-22e70b233590 admin admin] PATH_INFO: `/v3/projects/f8be60074ff941cd9db762708081cd10/users/e5fffda9d8e04d409cd12611b6682435/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:50.108055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e155-3769-4c9a-8704-22e70b233590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8be60074ff941cd9db762708081cd10, user_id=e5fffda9d8e04d409cd12611b6682435, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:50.117475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e155-3769-4c9a-8704-22e70b233590 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:50.254120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 846/1689] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:50 2022] PUT /identity/v3/projects/f8be60074ff941cd9db762708081cd10/users/e5fffda9d8e04d409cd12611b6682435/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:50.265796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c460b-eb24-45ce-81ec-73bef02ac2f2 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:50.266183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c460b-eb24-45ce-81ec-73bef02ac2f2 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:50.266559 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861c460b-eb24-45ce-81ec-73bef02ac2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:50.330225 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-861c460b-eb24-45ce-81ec-73bef02ac2f2 None None] MFA Rules not processed for user `e5fffda9d8e04d409cd12611b6682435`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:50.331530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-861c460b-eb24-45ce-81ec-73bef02ac2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:50.489316 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 844/1690] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:50.490561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650e155-3769-4c9a-8704-22e70b233590 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:50.642381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650e155-3769-4c9a-8704-22e70b233590 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:50.667711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d3a4f-ed25-4b79-aa38-2684e5c7ad9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:50.667711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d3a4f-ed25-4b79-aa38-2684e5c7ad9b admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:50.667711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d3a4f-ed25-4b79-aa38-2684e5c7ad9b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:50.667711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d3a4f-ed25-4b79-aa38-2684e5c7ad9b admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:50.668049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d3a4f-ed25-4b79-aa38-2684e5c7ad9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:50.668049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d3a4f-ed25-4b79-aa38-2684e5c7ad9b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:50.902393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 847/1691] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:50 2022] POST /identity/v3/projects => generated 370 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:50.906490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c460b-eb24-45ce-81ec-73bef02ac2f2 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:50.919813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861c460b-eb24-45ce-81ec-73bef02ac2f2 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:51.040924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff575c2d-eeb2-43bf-94e9-03cecfcfaa40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:51.041911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff575c2d-eeb2-43bf-94e9-03cecfcfaa40 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:51.042545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff575c2d-eeb2-43bf-94e9-03cecfcfaa40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:51.042976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff575c2d-eeb2-43bf-94e9-03cecfcfaa40 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:51.043829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff575c2d-eeb2-43bf-94e9-03cecfcfaa40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:51.050319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff575c2d-eeb2-43bf-94e9-03cecfcfaa40 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:51.188955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 845/1692] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:50 2022] POST /identity/v3/users => generated 292 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:51.194647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d3a4f-ed25-4b79-aa38-2684e5c7ad9b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:51.207831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d3a4f-ed25-4b79-aa38-2684e5c7ad9b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:51.215294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8daede2-678b-42bb-8739-678d28853b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:51.216232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daede2-678b-42bb-8739-678d28853b11 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:51.216653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daede2-678b-42bb-8739-678d28853b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:51.217183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8daede2-678b-42bb-8739-678d28853b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:51.218549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8daede2-678b-42bb-8739-678d28853b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:51.220819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8daede2-678b-42bb-8739-678d28853b11 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:51.235216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 848/1693] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:51 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:51.243441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff575c2d-eeb2-43bf-94e9-03cecfcfaa40 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:51.272231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff575c2d-eeb2-43bf-94e9-03cecfcfaa40 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:51.283572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2239a0-029f-4869-81a0-c7342c18f843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:51.284257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2239a0-029f-4869-81a0-c7342c18f843 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:51.284539 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2239a0-029f-4869-81a0-c7342c18f843 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:51.285041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2239a0-029f-4869-81a0-c7342c18f843 admin admin] PATH_INFO: `/v3/projects/8699853b99cb4e1dbebdf9e208e0542b/users/1257890081144963b55ee94e648d8228/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:51.350021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2239a0-029f-4869-81a0-c7342c18f843 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8699853b99cb4e1dbebdf9e208e0542b, user_id=1257890081144963b55ee94e648d8228, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:51.352960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2239a0-029f-4869-81a0-c7342c18f843 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:51.514211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 846/1694] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:51 2022] PUT /identity/v3/projects/8699853b99cb4e1dbebdf9e208e0542b/users/1257890081144963b55ee94e648d8228/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:51.518514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8daede2-678b-42bb-8739-678d28853b11 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:51.533766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8daede2-678b-42bb-8739-678d28853b11 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:51.660807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe275ac5-3de4-4da9-953b-87a79e47e056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:51.672251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe275ac5-3de4-4da9-953b-87a79e47e056 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:51.672251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe275ac5-3de4-4da9-953b-87a79e47e056 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:51.672251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe275ac5-3de4-4da9-953b-87a79e47e056 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:51.672251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe275ac5-3de4-4da9-953b-87a79e47e056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:51.672251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe275ac5-3de4-4da9-953b-87a79e47e056 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:51.686989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 849/1695] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:51 2022] GET /identity/v3/roles => generated 1427 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:59:51.692736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2239a0-029f-4869-81a0-c7342c18f843 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:51.713216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2239a0-029f-4869-81a0-c7342c18f843 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:51.742788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957636f3-64e9-4825-8031-70d40efdcb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:51.743546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957636f3-64e9-4825-8031-70d40efdcb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:51.743828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957636f3-64e9-4825-8031-70d40efdcb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:51.744102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957636f3-64e9-4825-8031-70d40efdcb94 admin admin] PATH_INFO: `/v3/projects/8699853b99cb4e1dbebdf9e208e0542b/users/1257890081144963b55ee94e648d8228/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:51.764065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957636f3-64e9-4825-8031-70d40efdcb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8699853b99cb4e1dbebdf9e208e0542b, user_id=1257890081144963b55ee94e648d8228, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:51.767031 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957636f3-64e9-4825-8031-70d40efdcb94 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:51.807447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 847/1696] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:51 2022] PUT /identity/v3/projects/8699853b99cb4e1dbebdf9e208e0542b/users/1257890081144963b55ee94e648d8228/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:51.818968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d98e3-5c97-4427-9f9c-4bf949daa3cf None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:51.819741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d98e3-5c97-4427-9f9c-4bf949daa3cf None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:51.820303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d98e3-5c97-4427-9f9c-4bf949daa3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:51.884931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-914d98e3-5c97-4427-9f9c-4bf949daa3cf None None] MFA Rules not processed for user `1257890081144963b55ee94e648d8228`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:51.886172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-914d98e3-5c97-4427-9f9c-4bf949daa3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:52.039654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 850/1697] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:52.047288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957636f3-64e9-4825-8031-70d40efdcb94 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:52.049999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-957636f3-64e9-4825-8031-70d40efdcb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:52.158850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:52.218117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957636f3-64e9-4825-8031-70d40efdcb94 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:52.230290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a83e8-f859-43d0-a5e0-048326b537bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:52.230926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a83e8-f859-43d0-a5e0-048326b537bb service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:52.231495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a83e8-f859-43d0-a5e0-048326b537bb service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:52.231911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a83e8-f859-43d0-a5e0-048326b537bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:52.267708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a83e8-f859-43d0-a5e0-048326b537bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:52.271293 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438a83e8-f859-43d0-a5e0-048326b537bb service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:52.287662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 848/1698] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:59:52.518447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:53.616428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c656f-3915-46d1-a652-e55b73b0d4b5 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:53.616992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c656f-3915-46d1-a652-e55b73b0d4b5 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:53.616992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c656f-3915-46d1-a652-e55b73b0d4b5 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:53.618709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 851/1699] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 04:59:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:59:53.747357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a83e8-f859-43d0-a5e0-048326b537bb service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:53.748982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-438a83e8-f859-43d0-a5e0-048326b537bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:53.762671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edf3f7-1868-4a4b-b5b9-55f9635570d1 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:53.763213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edf3f7-1868-4a4b-b5b9-55f9635570d1 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:53.763664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6edf3f7-1868-4a4b-b5b9-55f9635570d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:53.821787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d6edf3f7-1868-4a4b-b5b9-55f9635570d1 None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:53.826334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d6edf3f7-1868-4a4b-b5b9-55f9635570d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:53.948609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438a83e8-f859-43d0-a5e0-048326b537bb service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:53.961792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:53.982578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:53.985857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:53.986155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] PATH_INFO: `/v3/projects/46bf86c3f1894b9484ff64f8a76382fa` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:53.988597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46bf86c3f1894b9484ff64f8a76382fa)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:53.989590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: WARNING py.warnings [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 04:59:53.989590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: warnings.warn(msg) Aug 24 04:59:53.989590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: Aug 24 04:59:53.995610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:54.074344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 852/1700] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 04:59:53 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 04:59:54.401870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46bf86c3f1894b9484ff64f8a76382fa', 'request_id': 'req-92ea5f02-d37e-4870-a1e2-e8f32982ac04'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:54.441654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46bf86c3f1894b9484ff64f8a76382fa', 'request_id': 'req-92ea5f02-d37e-4870-a1e2-e8f32982ac04'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:54.626899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46bf86c3f1894b9484ff64f8a76382fa', 'request_id': 'req-92ea5f02-d37e-4870-a1e2-e8f32982ac04'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:54.650998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] The token cache is being invalidate because project 46bf86c3f1894b9484ff64f8a76382fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:59:54.653276 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:54.656036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:54.658377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 849/1701] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 04:59:53 2022] DELETE /identity/v3/projects/46bf86c3f1894b9484ff64f8a76382fa => generated 0 bytes in 912 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:54.664309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edf3f7-1868-4a4b-b5b9-55f9635570d1 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:54.690092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6edf3f7-1868-4a4b-b5b9-55f9635570d1 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:54.892916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:54.894295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:54.894873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:54.895525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] PATH_INFO: `/v3/users/5d7dc14814784102978d2930604c956f` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:54.897119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d7dc14814784102978d2930604c956f)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:54.899971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:54.920722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:55.063496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:55.065605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:55.145010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:55.180946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ea5f02-d37e-4870-a1e2-e8f32982ac04 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:55.189365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbe398-1942-429e-9b73-105b377255b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:55.189365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe398-1942-429e-9b73-105b377255b6 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:55.189365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe398-1942-429e-9b73-105b377255b6 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:55.198580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbe398-1942-429e-9b73-105b377255b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:55.203359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e2fbe398-1942-429e-9b73-105b377255b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:55.267239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d7dc14814784102978d2930604c956f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:55.270356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d7dc14814784102978d2930604c956f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:55.274515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d7dc14814784102978d2930604c956f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:55.291293 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe398-1942-429e-9b73-105b377255b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:55.303735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbe398-1942-429e-9b73-105b377255b6 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:55.334281 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 850/1702] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:55 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:55.339511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f335e6a-ed15-4d55-8792-318d446661b3 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:55.340036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f335e6a-ed15-4d55-8792-318d446661b3 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:55.340817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f335e6a-ed15-4d55-8792-318d446661b3 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:55.343735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 851/1703] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 04:59:55 2022] GET /identity => generated 271 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 04:59:55.421399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d7dc14814784102978d2930604c956f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:55.478638 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d7dc14814784102978d2930604c956f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:55.548896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 853/1704] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 04:59:54 2022] DELETE /identity/v3/users/5d7dc14814784102978d2930604c956f => generated 0 bytes in 885 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:55.573659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f130b3-75c5-4e5d-adb3-9dab8ba94969 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:55.574238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f130b3-75c5-4e5d-adb3-9dab8ba94969 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:55.574616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f130b3-75c5-4e5d-adb3-9dab8ba94969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:55.648345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-37f130b3-75c5-4e5d-adb3-9dab8ba94969 None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:55.650836 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-37f130b3-75c5-4e5d-adb3-9dab8ba94969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:55.882961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 852/1705] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 04:59:55 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:56.210469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:56.213915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:56.248852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c3166-144d-4c68-83f1-c4bbc663cdec admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:56.267658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412c31e-98d6-4dc1-a035-f6d543a44d1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:56.269428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412c31e-98d6-4dc1-a035-f6d543a44d1d service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:56.270131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412c31e-98d6-4dc1-a035-f6d543a44d1d service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:56.270660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412c31e-98d6-4dc1-a035-f6d543a44d1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:56.293586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412c31e-98d6-4dc1-a035-f6d543a44d1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:56.295972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412c31e-98d6-4dc1-a035-f6d543a44d1d service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:56.312208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 854/1706] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:56 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:59:57.339648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f130b3-75c5-4e5d-adb3-9dab8ba94969 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:57.341667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-37f130b3-75c5-4e5d-adb3-9dab8ba94969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:57.495429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f130b3-75c5-4e5d-adb3-9dab8ba94969 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:57.502453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:57.504571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:57.505119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:57.505400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] PATH_INFO: `/v3/projects/4caaae6efb3f421e923b8050fddcd15a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:57.506748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4caaae6efb3f421e923b8050fddcd15a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:57.515320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:57.747994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4caaae6efb3f421e923b8050fddcd15a', 'request_id': 'req-6b028594-21de-421a-a192-a270f3c8f667'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:57.768631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4caaae6efb3f421e923b8050fddcd15a', 'request_id': 'req-6b028594-21de-421a-a192-a270f3c8f667'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:57.953419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4caaae6efb3f421e923b8050fddcd15a', 'request_id': 'req-6b028594-21de-421a-a192-a270f3c8f667'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:57.975470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] The token cache is being invalidate because project 4caaae6efb3f421e923b8050fddcd15a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:59:57.975768 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:57.979066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6b028594-21de-421a-a192-a270f3c8f667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:57.981960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 853/1707] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 04:59:57 2022] DELETE /identity/v3/projects/4caaae6efb3f421e923b8050fddcd15a => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:59:57.987149 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539436d-0314-494f-a495-8e9fbd927b8b None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:57.988114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539436d-0314-494f-a495-8e9fbd927b8b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:57.988599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539436d-0314-494f-a495-8e9fbd927b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:58.043555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-e539436d-0314-494f-a495-8e9fbd927b8b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:58.044900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e539436d-0314-494f-a495-8e9fbd927b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:58.171117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 855/1708] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:58.171868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507285-b0fb-4717-b951-b932af736920 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:58.172840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507285-b0fb-4717-b951-b932af736920 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:58.173261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85507285-b0fb-4717-b951-b932af736920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:58.218483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-85507285-b0fb-4717-b951-b932af736920 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:58.219660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-85507285-b0fb-4717-b951-b932af736920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:58.242517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539436d-0314-494f-a495-8e9fbd927b8b None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:58.250958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 854/1709] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:58.260970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85507285-b0fb-4717-b951-b932af736920 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:58.261983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539436d-0314-494f-a495-8e9fbd927b8b None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:58.263986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-85507285-b0fb-4717-b951-b932af736920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:58.297085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3da10-35b8-4b1d-b154-f75a3da5aec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:58.297085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3da10-35b8-4b1d-b154-f75a3da5aec1 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:58.297085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3da10-35b8-4b1d-b154-f75a3da5aec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:58.297085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a3da10-35b8-4b1d-b154-f75a3da5aec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:58.297797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3da10-35b8-4b1d-b154-f75a3da5aec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 04:59:58.297797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3da10-35b8-4b1d-b154-f75a3da5aec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:58.297797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a3da10-35b8-4b1d-b154-f75a3da5aec1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:58.309669 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85507285-b0fb-4717-b951-b932af736920 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:58.328328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:58.333883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:58.333883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:58.333883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] PATH_INFO: `/v3/users/e5fffda9d8e04d409cd12611b6682435` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:58.333883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5fffda9d8e04d409cd12611b6682435)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:58.339113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:58.341651 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 856/1710] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 04:59:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 04:59:58.348295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3da10-35b8-4b1d-b154-f75a3da5aec1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:58.386407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a3da10-35b8-4b1d-b154-f75a3da5aec1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:58.395130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1d1e7-07ea-422d-9729-c3d0c0a54ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:58.396020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1d1e7-07ea-422d-9729-c3d0c0a54ede admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:58.396478 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1d1e7-07ea-422d-9729-c3d0c0a54ede admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:58.396919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1d1e7-07ea-422d-9729-c3d0c0a54ede admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:58.397751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1d1e7-07ea-422d-9729-c3d0c0a54ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:58.400092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e1d1e7-07ea-422d-9729-c3d0c0a54ede admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:58.685359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 857/1711] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:58 2022] POST /identity/v3/projects => generated 388 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:58.689877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1d1e7-07ea-422d-9729-c3d0c0a54ede admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:58.716363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5fffda9d8e04d409cd12611b6682435'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:58.717887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5fffda9d8e04d409cd12611b6682435'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:58.720786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5fffda9d8e04d409cd12611b6682435'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:58.808782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5fffda9d8e04d409cd12611b6682435'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:58.831531 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fc35bf86-c60b-4f83-923e-86e07bf60d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5fffda9d8e04d409cd12611b6682435'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:59:58.847317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:58.891388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 855/1712] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 04:59:58 2022] DELETE /identity/v3/users/e5fffda9d8e04d409cd12611b6682435 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:59:58.899886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991061d-b9ba-4381-ac93-d54bf4596b54 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:58.901687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991061d-b9ba-4381-ac93-d54bf4596b54 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:58.903267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991061d-b9ba-4381-ac93-d54bf4596b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:58.914904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1d1e7-07ea-422d-9729-c3d0c0a54ede admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:58.967721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2991061d-b9ba-4381-ac93-d54bf4596b54 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:58.970424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2991061d-b9ba-4381-ac93-d54bf4596b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:59.056137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ea9d7-a254-4917-b0f8-dea4ec2f80e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:59.057399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ea9d7-a254-4917-b0f8-dea4ec2f80e4 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:59.057749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ea9d7-a254-4917-b0f8-dea4ec2f80e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:59.058030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ea9d7-a254-4917-b0f8-dea4ec2f80e4 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:59.058661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ea9d7-a254-4917-b0f8-dea4ec2f80e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:59.060797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ea9d7-a254-4917-b0f8-dea4ec2f80e4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:59.115880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 856/1713] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:59.151581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991061d-b9ba-4381-ac93-d54bf4596b54 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:59.153459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2991061d-b9ba-4381-ac93-d54bf4596b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:59.232653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 858/1714] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 04:59:58 2022] POST /identity/v3/users => generated 295 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 04:59:59.236851 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ea9d7-a254-4917-b0f8-dea4ec2f80e4 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:59.242025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ed6ea9d7-a254-4917-b0f8-dea4ec2f80e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:59.275162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ea9d7-a254-4917-b0f8-dea4ec2f80e4 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:59.311379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4f01d-889b-49f5-9b43-00ee4097ca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:59.315180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4f01d-889b-49f5-9b43-00ee4097ca70 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:59.315638 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4f01d-889b-49f5-9b43-00ee4097ca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:59.324058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4f01d-889b-49f5-9b43-00ee4097ca70 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:59.324058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4f01d-889b-49f5-9b43-00ee4097ca70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:59.328763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4f01d-889b-49f5-9b43-00ee4097ca70 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:59.343688 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991061d-b9ba-4381-ac93-d54bf4596b54 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:59.357434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f7549-9883-486b-837f-a0e271971925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:59.358345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f7549-9883-486b-837f-a0e271971925 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:59.359681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f7549-9883-486b-837f-a0e271971925 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:59.360097 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f7549-9883-486b-837f-a0e271971925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:59.365495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 859/1715] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:59 2022] GET /identity/v3/roles => generated 1427 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 04:59:59.369812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4f01d-889b-49f5-9b43-00ee4097ca70 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:59.395785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f7549-9883-486b-837f-a0e271971925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:59.400543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4f01d-889b-49f5-9b43-00ee4097ca70 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:59.401658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3f7549-9883-486b-837f-a0e271971925 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:59.422722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f82c0e-0a94-4f51-9897-500250f77419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:59.423648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f82c0e-0a94-4f51-9897-500250f77419 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:59.426490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f82c0e-0a94-4f51-9897-500250f77419 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:59.428659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f82c0e-0a94-4f51-9897-500250f77419 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:59.429499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f82c0e-0a94-4f51-9897-500250f77419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:59.431543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f82c0e-0a94-4f51-9897-500250f77419 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:59.434396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 857/1716] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 04:59:59 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 04:59:59.461530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 860/1717] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 04:59:59 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 04:59:59.465132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f7549-9883-486b-837f-a0e271971925 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:59:59.485656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f7549-9883-486b-837f-a0e271971925 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:59:59.491798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff104b-0eab-41da-8268-3d0ab2d521c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:59:59.492997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff104b-0eab-41da-8268-3d0ab2d521c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:59.493477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff104b-0eab-41da-8268-3d0ab2d521c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:59.493916 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff104b-0eab-41da-8268-3d0ab2d521c7 admin admin] PATH_INFO: `/v3/projects/dca24fb48ad842e6b12efd85726dd82a/users/7d1603851b7d4baabf48b3fac47433de/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:59.552783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff104b-0eab-41da-8268-3d0ab2d521c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dca24fb48ad842e6b12efd85726dd82a, user_id=7d1603851b7d4baabf48b3fac47433de, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:59:59.555655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff104b-0eab-41da-8268-3d0ab2d521c7 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:59:59.738347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 858/1718] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 04:59:59 2022] PUT /identity/v3/projects/dca24fb48ad842e6b12efd85726dd82a/users/7d1603851b7d4baabf48b3fac47433de/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 04:59:59.749158 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ce43-f3aa-4ba2-99af-5a88367225fe None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:59:59.749467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ce43-f3aa-4ba2-99af-5a88367225fe None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:59:59.749755 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ce43-f3aa-4ba2-99af-5a88367225fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:59:59.790682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-4f08ce43-f3aa-4ba2-99af-5a88367225fe None None] MFA Rules not processed for user `7d1603851b7d4baabf48b3fac47433de`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 04:59:59.800638 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4f08ce43-f3aa-4ba2-99af-5a88367225fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:59:59.872887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:59:59.947339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 861/1719] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 04:59:59 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 04:59:59.951954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff104b-0eab-41da-8268-3d0ab2d521c7 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:00.127211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff104b-0eab-41da-8268-3d0ab2d521c7 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:00.133978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04589e9-c11b-4741-9474-934fdde98e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:00.135572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04589e9-c11b-4741-9474-934fdde98e3f admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:00.136208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04589e9-c11b-4741-9474-934fdde98e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:00.136615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04589e9-c11b-4741-9474-934fdde98e3f admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:00.142289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04589e9-c11b-4741-9474-934fdde98e3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:00.146609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04589e9-c11b-4741-9474-934fdde98e3f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:00.375378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 859/1720] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 04:59:59 2022] POST /identity/v3/projects => generated 388 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:00.381933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08ce43-f3aa-4ba2-99af-5a88367225fe None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:00.398392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08ce43-f3aa-4ba2-99af-5a88367225fe None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:00.531455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ea3b0-3fcc-4aa1-bd29-6975cfbce8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:00.532207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ea3b0-3fcc-4aa1-bd29-6975cfbce8f5 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:00.532580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ea3b0-3fcc-4aa1-bd29-6975cfbce8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:00.533086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ea3b0-3fcc-4aa1-bd29-6975cfbce8f5 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:00.534095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ea3b0-3fcc-4aa1-bd29-6975cfbce8f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:00.536622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ea3b0-3fcc-4aa1-bd29-6975cfbce8f5 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:00.662953 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 862/1721] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:00 2022] POST /identity/v3/users => generated 295 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:00:00.669034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04589e9-c11b-4741-9474-934fdde98e3f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:00.684195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04589e9-c11b-4741-9474-934fdde98e3f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:00.691946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7aa7f-b8e4-41ff-9081-1227554f1e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:00.692998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7aa7f-b8e4-41ff-9081-1227554f1e4c admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:00.693661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7aa7f-b8e4-41ff-9081-1227554f1e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:00.694212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7aa7f-b8e4-41ff-9081-1227554f1e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:00.695591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7aa7f-b8e4-41ff-9081-1227554f1e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:00.697984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7aa7f-b8e4-41ff-9081-1227554f1e4c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:00.713857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 860/1722] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:00 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:00.718046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ea3b0-3fcc-4aa1-bd29-6975cfbce8f5 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:00.730627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ea3b0-3fcc-4aa1-bd29-6975cfbce8f5 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:00.736395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9357004-3835-4906-84de-c5a0ddfc2c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:00.737950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9357004-3835-4906-84de-c5a0ddfc2c41 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:00.738712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9357004-3835-4906-84de-c5a0ddfc2c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:00.739585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9357004-3835-4906-84de-c5a0ddfc2c41 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:00.740412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9357004-3835-4906-84de-c5a0ddfc2c41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:00.742914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9357004-3835-4906-84de-c5a0ddfc2c41 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:00.755069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 863/1723] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:00 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:00.759176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7aa7f-b8e4-41ff-9081-1227554f1e4c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:00.771298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7aa7f-b8e4-41ff-9081-1227554f1e4c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:00.780283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47471678-78f0-4853-ad67-51cf58c7ef3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:00.785717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47471678-78f0-4853-ad67-51cf58c7ef3b admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:00.785717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47471678-78f0-4853-ad67-51cf58c7ef3b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:00.785717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47471678-78f0-4853-ad67-51cf58c7ef3b admin admin] PATH_INFO: `/v3/projects/0548b560bb5f45a69d87fe348558a505/users/786077efd1fa4e598d53a905dadcb6cb/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:00.801585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47471678-78f0-4853-ad67-51cf58c7ef3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0548b560bb5f45a69d87fe348558a505, user_id=786077efd1fa4e598d53a905dadcb6cb, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:00.804548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47471678-78f0-4853-ad67-51cf58c7ef3b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:00.942927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 861/1724] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:00 2022] PUT /identity/v3/projects/0548b560bb5f45a69d87fe348558a505/users/786077efd1fa4e598d53a905dadcb6cb/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:00.953643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123db19-9ab4-4de7-9d7b-d444deadde92 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:00.954368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123db19-9ab4-4de7-9d7b-d444deadde92 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:00.954880 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123db19-9ab4-4de7-9d7b-d444deadde92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:00.986600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d123db19-9ab4-4de7-9d7b-d444deadde92 None None] MFA Rules not processed for user `786077efd1fa4e598d53a905dadcb6cb`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:00.988444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d123db19-9ab4-4de7-9d7b-d444deadde92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:01.001719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47471678-78f0-4853-ad67-51cf58c7ef3b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:01.004585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-47471678-78f0-4853-ad67-51cf58c7ef3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:01.092726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 864/1725] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:00 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:00:01.098349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123db19-9ab4-4de7-9d7b-d444deadde92 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:01.098513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47471678-78f0-4853-ad67-51cf58c7ef3b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:01.103421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:01.104010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:01.104286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:01.104563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] PATH_INFO: `/v3/projects/f8be60074ff941cd9db762708081cd10` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:01.105803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8be60074ff941cd9db762708081cd10)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:01.107784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:01.116173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123db19-9ab4-4de7-9d7b-d444deadde92 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:01.121683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f142c2c-6a3b-44bb-937a-ee088a6607a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:01.122300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f142c2c-6a3b-44bb-937a-ee088a6607a9 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:01.122578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f142c2c-6a3b-44bb-937a-ee088a6607a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:01.122873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f142c2c-6a3b-44bb-937a-ee088a6607a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:01.123539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f142c2c-6a3b-44bb-937a-ee088a6607a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:01.125456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f142c2c-6a3b-44bb-937a-ee088a6607a9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:01.268227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 865/1726] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:01 2022] POST /identity/v3/projects => generated 388 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:01.271849 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f142c2c-6a3b-44bb-937a-ee088a6607a9 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:01.302277 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8be60074ff941cd9db762708081cd10', 'request_id': 'req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:01.309105 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8be60074ff941cd9db762708081cd10', 'request_id': 'req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:01.335175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f142c2c-6a3b-44bb-937a-ee088a6607a9 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:01.394618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553aa88-d25f-4196-928b-b18bf7c3057f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:01.395293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553aa88-d25f-4196-928b-b18bf7c3057f admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:01.395634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553aa88-d25f-4196-928b-b18bf7c3057f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:01.396082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553aa88-d25f-4196-928b-b18bf7c3057f admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:01.396714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553aa88-d25f-4196-928b-b18bf7c3057f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:01.398695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553aa88-d25f-4196-928b-b18bf7c3057f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:01.436741 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8be60074ff941cd9db762708081cd10', 'request_id': 'req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:01.449394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] The token cache is being invalidate because project f8be60074ff941cd9db762708081cd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:01.449815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:01.451150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:01.452998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 862/1727] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:01 2022] DELETE /identity/v3/projects/f8be60074ff941cd9db762708081cd10 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:01.463891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:01.465763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:01.480648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0008a3-b01e-43fd-b9b8-948c0e9a5a8e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:01.485502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:01.486206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:01.486574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:01.487458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] PATH_INFO: `/v3/users/1257890081144963b55ee94e648d8228` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:01.489123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1257890081144963b55ee94e648d8228)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:01.491209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:01.544177 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 866/1728] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:01 2022] POST /identity/v3/users => generated 301 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:00:01.545930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553aa88-d25f-4196-928b-b18bf7c3057f admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:01.547680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a553aa88-d25f-4196-928b-b18bf7c3057f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:01.561460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553aa88-d25f-4196-928b-b18bf7c3057f admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:01.566278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e2572-a00b-4d02-8113-e8ac1dd1b4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:01.567093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e2572-a00b-4d02-8113-e8ac1dd1b4aa admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:01.567554 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e2572-a00b-4d02-8113-e8ac1dd1b4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:01.567724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e2572-a00b-4d02-8113-e8ac1dd1b4aa admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:01.568397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e2572-a00b-4d02-8113-e8ac1dd1b4aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:01.570065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e2572-a00b-4d02-8113-e8ac1dd1b4aa admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:01.584896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 867/1729] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:01 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:00:01.589077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e2572-a00b-4d02-8113-e8ac1dd1b4aa admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:01.605310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e2572-a00b-4d02-8113-e8ac1dd1b4aa admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:01.610501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed693f66-40d7-4d00-95c6-8c84af9b50ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:01.611201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed693f66-40d7-4d00-95c6-8c84af9b50ba admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:01.611487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed693f66-40d7-4d00-95c6-8c84af9b50ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:01.612421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed693f66-40d7-4d00-95c6-8c84af9b50ba admin admin] PATH_INFO: `/v3/projects/46d018ad4bda44268f69c9c065e96366/users/1c67e25381974886a89d25b7a420b3c5/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:01.629063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed693f66-40d7-4d00-95c6-8c84af9b50ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d018ad4bda44268f69c9c065e96366, user_id=1c67e25381974886a89d25b7a420b3c5, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:01.631907 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed693f66-40d7-4d00-95c6-8c84af9b50ba admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:01.805075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 868/1730] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:01 2022] PUT /identity/v3/projects/46d018ad4bda44268f69c9c065e96366/users/1c67e25381974886a89d25b7a420b3c5/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:01.809345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed693f66-40d7-4d00-95c6-8c84af9b50ba admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:01.813127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1257890081144963b55ee94e648d8228'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:01.813777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1257890081144963b55ee94e648d8228'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:01.814383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1257890081144963b55ee94e648d8228'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:01.871217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed693f66-40d7-4d00-95c6-8c84af9b50ba admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:01.875860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe190924-e38e-4968-9b72-af5a7a0d74dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:01.876394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe190924-e38e-4968-9b72-af5a7a0d74dc admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:01.876665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe190924-e38e-4968-9b72-af5a7a0d74dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:01.876945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe190924-e38e-4968-9b72-af5a7a0d74dc admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:01.877717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe190924-e38e-4968-9b72-af5a7a0d74dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:01.879556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe190924-e38e-4968-9b72-af5a7a0d74dc admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:01.889076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 869/1731] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:01 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:00:01.893585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe190924-e38e-4968-9b72-af5a7a0d74dc admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:01.895159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fe190924-e38e-4968-9b72-af5a7a0d74dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:01.907178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe190924-e38e-4968-9b72-af5a7a0d74dc admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:01.912294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1328721-748b-4ab0-ad15-3294612d81a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:01.912909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1328721-748b-4ab0-ad15-3294612d81a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:01.913183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1328721-748b-4ab0-ad15-3294612d81a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:01.913452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1328721-748b-4ab0-ad15-3294612d81a9 admin admin] PATH_INFO: `/v3/projects/46d018ad4bda44268f69c9c065e96366/users/1c67e25381974886a89d25b7a420b3c5/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:01.915324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1328721-748b-4ab0-ad15-3294612d81a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d018ad4bda44268f69c9c065e96366, user_id=1c67e25381974886a89d25b7a420b3c5, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:01.917908 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1328721-748b-4ab0-ad15-3294612d81a9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:01.927885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 870/1732] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:01 2022] PUT /identity/v3/projects/46d018ad4bda44268f69c9c065e96366/users/1c67e25381974886a89d25b7a420b3c5/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:01.933505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1257890081144963b55ee94e648d8228'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:01.939333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76638bf-a7bf-4a3f-9992-515b572f9eae None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:01.941626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76638bf-a7bf-4a3f-9992-515b572f9eae None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:01.941626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76638bf-a7bf-4a3f-9992-515b572f9eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:01.946956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1257890081144963b55ee94e648d8228'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:01.959311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 863/1733] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:01 2022] DELETE /identity/v3/users/1257890081144963b55ee94e648d8228 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:01.968083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f76638bf-a7bf-4a3f-9992-515b572f9eae None None] MFA Rules not processed for user `1c67e25381974886a89d25b7a420b3c5`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:01.969300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f76638bf-a7bf-4a3f-9992-515b572f9eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:02.037692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 871/1734] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:01 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:00:02.048822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:02.050724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:02.107590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a76dd-e086-41dd-a871-8537cdbc0140 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:02.111218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03ea2aa-65d5-489c-9b73-e822433011a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:02.111870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ea2aa-65d5-489c-9b73-e822433011a4 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:02.112146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ea2aa-65d5-489c-9b73-e822433011a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:02.112421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ea2aa-65d5-489c-9b73-e822433011a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:02.121631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ea2aa-65d5-489c-9b73-e822433011a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:02.123073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03ea2aa-65d5-489c-9b73-e822433011a4 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:02.132550 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 864/1735] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:02 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:02.615547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76638bf-a7bf-4a3f-9992-515b572f9eae None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:02.617227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f76638bf-a7bf-4a3f-9992-515b572f9eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:02.715887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76638bf-a7bf-4a3f-9992-515b572f9eae None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:02.722288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:02.722948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:02.723257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:02.723532 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] PATH_INFO: `/v3/projects/8699853b99cb4e1dbebdf9e208e0542b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:02.724780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8699853b99cb4e1dbebdf9e208e0542b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:02.726947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:02.929579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8699853b99cb4e1dbebdf9e208e0542b', 'request_id': 'req-643b09d9-fb25-4c91-a400-ea7752161166'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:02.936166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8699853b99cb4e1dbebdf9e208e0542b', 'request_id': 'req-643b09d9-fb25-4c91-a400-ea7752161166'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:02.942700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8699853b99cb4e1dbebdf9e208e0542b', 'request_id': 'req-643b09d9-fb25-4c91-a400-ea7752161166'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:03.059363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:03.067384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] The token cache is being invalidate because project 8699853b99cb4e1dbebdf9e208e0542b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:03.067792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:03.068437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-643b09d9-fb25-4c91-a400-ea7752161166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:03.069950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 872/1736] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:02 2022] DELETE /identity/v3/projects/8699853b99cb4e1dbebdf9e208e0542b => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:03.074761 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b798e-7978-445e-bf57-0a22d9c97a85 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:03.075250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b798e-7978-445e-bf57-0a22d9c97a85 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:03.075350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b798e-7978-445e-bf57-0a22d9c97a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:03.094288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-981b798e-7978-445e-bf57-0a22d9c97a85 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:03.104887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-981b798e-7978-445e-bf57-0a22d9c97a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:03.156067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 865/1737] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:03.165187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ecda-5840-4112-bb52-458f4cdf3e5b None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:03.165473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ecda-5840-4112-bb52-458f4cdf3e5b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:03.166058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90ecda-5840-4112-bb52-458f4cdf3e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:03.192903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-fd90ecda-5840-4112-bb52-458f4cdf3e5b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:03.193752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fd90ecda-5840-4112-bb52-458f4cdf3e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:03.208750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 873/1738] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:03.208870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b798e-7978-445e-bf57-0a22d9c97a85 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:03.218698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b798e-7978-445e-bf57-0a22d9c97a85 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:03.224430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86b58-9fe3-4698-a293-6ec1d98f6f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:03.225324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86b58-9fe3-4698-a293-6ec1d98f6f28 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:03.225912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86b58-9fe3-4698-a293-6ec1d98f6f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:03.226422 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd86b58-9fe3-4698-a293-6ec1d98f6f28 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:03.227313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd86b58-9fe3-4698-a293-6ec1d98f6f28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:00:03.227914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd86b58-9fe3-4698-a293-6ec1d98f6f28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:03.229937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd86b58-9fe3-4698-a293-6ec1d98f6f28 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:03.244074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 866/1739] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:00:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:00:03.248535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90ecda-5840-4112-bb52-458f4cdf3e5b None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:03.258469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90ecda-5840-4112-bb52-458f4cdf3e5b None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:03.263206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab8350-2074-4cf9-b430-0ee77b75c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:03.263814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab8350-2074-4cf9-b430-0ee77b75c507 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:03.264090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab8350-2074-4cf9-b430-0ee77b75c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:03.264367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab8350-2074-4cf9-b430-0ee77b75c507 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:03.264973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab8350-2074-4cf9-b430-0ee77b75c507 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:03.266891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab8350-2074-4cf9-b430-0ee77b75c507 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:03.465961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 874/1740] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:03 2022] POST /identity/v3/projects => generated 384 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:03.467813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86b58-9fe3-4698-a293-6ec1d98f6f28 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:03.500799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd86b58-9fe3-4698-a293-6ec1d98f6f28 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:03.590429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1dbdb-0cb3-4a72-ac20-c0cdd68c1e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:03.591222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1dbdb-0cb3-4a72-ac20-c0cdd68c1e6a admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:03.591556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1dbdb-0cb3-4a72-ac20-c0cdd68c1e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:03.592285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1dbdb-0cb3-4a72-ac20-c0cdd68c1e6a admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:03.592745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1dbdb-0cb3-4a72-ac20-c0cdd68c1e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:03.598447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1dbdb-0cb3-4a72-ac20-c0cdd68c1e6a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:03.679229 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:03.845243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 867/1741] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:03 2022] POST /identity/v3/users => generated 293 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:03.849982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab8350-2074-4cf9-b430-0ee77b75c507 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:03.860986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab8350-2074-4cf9-b430-0ee77b75c507 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:03.866973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce4748-1a6c-45da-9a54-8e847b8c0345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:03.867776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce4748-1a6c-45da-9a54-8e847b8c0345 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:03.868202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce4748-1a6c-45da-9a54-8e847b8c0345 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:03.868618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce4748-1a6c-45da-9a54-8e847b8c0345 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:03.869422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce4748-1a6c-45da-9a54-8e847b8c0345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:03.871577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce4748-1a6c-45da-9a54-8e847b8c0345 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:03.879053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1dbdb-0cb3-4a72-ac20-c0cdd68c1e6a admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:03.880789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-edf1dbdb-0cb3-4a72-ac20-c0cdd68c1e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:03.886904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 875/1742] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:03 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:03.897233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce4748-1a6c-45da-9a54-8e847b8c0345 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:03.919901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce4748-1a6c-45da-9a54-8e847b8c0345 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:03.929786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b64cd7-9fd3-44f9-8bf5-9e293dee20ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:03.930923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b64cd7-9fd3-44f9-8bf5-9e293dee20ac admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:03.931377 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b64cd7-9fd3-44f9-8bf5-9e293dee20ac admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:03.931798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b64cd7-9fd3-44f9-8bf5-9e293dee20ac admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:03.932727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b64cd7-9fd3-44f9-8bf5-9e293dee20ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:03.934630 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b64cd7-9fd3-44f9-8bf5-9e293dee20ac admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:03.949962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 876/1743] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:03 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:03.950942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b64cd7-9fd3-44f9-8bf5-9e293dee20ac admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:03.961958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b64cd7-9fd3-44f9-8bf5-9e293dee20ac admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:03.966857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed148-8130-4c71-bc24-abfc201daae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:03.967588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed148-8130-4c71-bc24-abfc201daae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:03.968505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed148-8130-4c71-bc24-abfc201daae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:03.968505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed148-8130-4c71-bc24-abfc201daae4 admin admin] PATH_INFO: `/v3/projects/4f3ae1939c0247308ad975d2690c5516/users/13e3f84752dc44b79a1025c2f0ea9882/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:03.974134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1dbdb-0cb3-4a72-ac20-c0cdd68c1e6a admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:03.979170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1795607-5e08-49c8-bc09-1f9caf14c376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:03.979900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1795607-5e08-49c8-bc09-1f9caf14c376 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:03.980420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1795607-5e08-49c8-bc09-1f9caf14c376 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:03.980766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1795607-5e08-49c8-bc09-1f9caf14c376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:03.983527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d1795607-5e08-49c8-bc09-1f9caf14c376 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:03.984106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ed148-8130-4c71-bc24-abfc201daae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3ae1939c0247308ad975d2690c5516, user_id=13e3f84752dc44b79a1025c2f0ea9882, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:03.987088 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ed148-8130-4c71-bc24-abfc201daae4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:04.051414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1795607-5e08-49c8-bc09-1f9caf14c376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:04.053006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1795607-5e08-49c8-bc09-1f9caf14c376 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:04.064254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 868/1744] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:03 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:00:04.086580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 877/1745] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:03 2022] PUT /identity/v3/projects/4f3ae1939c0247308ad975d2690c5516/users/13e3f84752dc44b79a1025c2f0ea9882/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:04.089096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71bbfe-0367-4499-969f-91b2dd0cf76a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:04.094797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71bbfe-0367-4499-969f-91b2dd0cf76a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:04.094797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71bbfe-0367-4499-969f-91b2dd0cf76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:04.129020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2a71bbfe-0367-4499-969f-91b2dd0cf76a None None] MFA Rules not processed for user `13e3f84752dc44b79a1025c2f0ea9882`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:04.130583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2a71bbfe-0367-4499-969f-91b2dd0cf76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:04.212675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 869/1746] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:04.212765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed148-8130-4c71-bc24-abfc201daae4 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:04.276142 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed148-8130-4c71-bc24-abfc201daae4 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:04.280749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e70574-af5b-4291-a24d-791cc8f9b753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:04.281411 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e70574-af5b-4291-a24d-791cc8f9b753 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:04.281687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e70574-af5b-4291-a24d-791cc8f9b753 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:04.281959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e70574-af5b-4291-a24d-791cc8f9b753 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:04.282636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e70574-af5b-4291-a24d-791cc8f9b753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:04.285194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e70574-af5b-4291-a24d-791cc8f9b753 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:04.416347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 878/1747] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:04 2022] POST /identity/v3/projects => generated 388 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:00:04.420747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71bbfe-0367-4499-969f-91b2dd0cf76a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:04.432242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71bbfe-0367-4499-969f-91b2dd0cf76a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:04.496085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f83a89-7fe1-432c-84d9-e8451f59089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:04.496621 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f83a89-7fe1-432c-84d9-e8451f59089e admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:04.496902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f83a89-7fe1-432c-84d9-e8451f59089e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:04.497177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f83a89-7fe1-432c-84d9-e8451f59089e admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:04.497804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f83a89-7fe1-432c-84d9-e8451f59089e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:04.499799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f83a89-7fe1-432c-84d9-e8451f59089e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:04.792014 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 870/1748] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:04 2022] POST /identity/v3/users => generated 301 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:04.796244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e70574-af5b-4291-a24d-791cc8f9b753 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:04.807260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e70574-af5b-4291-a24d-791cc8f9b753 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:04.811957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0945a-dcd3-49f3-8179-60eaaeeea33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:04.812563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0945a-dcd3-49f3-8179-60eaaeeea33f admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:04.812922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0945a-dcd3-49f3-8179-60eaaeeea33f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:04.813220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0945a-dcd3-49f3-8179-60eaaeeea33f admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:04.813852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0945a-dcd3-49f3-8179-60eaaeeea33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:04.816292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0945a-dcd3-49f3-8179-60eaaeeea33f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:04.827286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 879/1749] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:04 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:00:04.833826 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f83a89-7fe1-432c-84d9-e8451f59089e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:04.844112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f83a89-7fe1-432c-84d9-e8451f59089e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:04.848931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f48408-8c4d-4f8a-a0d1-2f39a4c55c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:04.849591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f48408-8c4d-4f8a-a0d1-2f39a4c55c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:04.849865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f48408-8c4d-4f8a-a0d1-2f39a4c55c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:04.850134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f48408-8c4d-4f8a-a0d1-2f39a4c55c7c admin admin] PATH_INFO: `/v3/projects/cf863097bc4448e081e1b5c83eedf894/users/0471ee8d2b9345d2b86487e96b70013e/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:04.865687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f48408-8c4d-4f8a-a0d1-2f39a4c55c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf863097bc4448e081e1b5c83eedf894, user_id=0471ee8d2b9345d2b86487e96b70013e, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:04.868612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f48408-8c4d-4f8a-a0d1-2f39a4c55c7c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:05.019804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 871/1750] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:04 2022] PUT /identity/v3/projects/cf863097bc4448e081e1b5c83eedf894/users/0471ee8d2b9345d2b86487e96b70013e/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:05.023960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0945a-dcd3-49f3-8179-60eaaeeea33f admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:05.035084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0945a-dcd3-49f3-8179-60eaaeeea33f admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:05.109064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac7d68-0cc8-469c-bd62-81d0b54027d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:05.109064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac7d68-0cc8-469c-bd62-81d0b54027d8 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:05.109064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac7d68-0cc8-469c-bd62-81d0b54027d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:05.109064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac7d68-0cc8-469c-bd62-81d0b54027d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:05.109995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac7d68-0cc8-469c-bd62-81d0b54027d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:05.109995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac7d68-0cc8-469c-bd62-81d0b54027d8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:05.120774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 880/1751] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:05 2022] GET /identity/v3/roles => generated 1427 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:05.125534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f48408-8c4d-4f8a-a0d1-2f39a4c55c7c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:05.137861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f48408-8c4d-4f8a-a0d1-2f39a4c55c7c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:05.142482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd12e9-513d-478f-9e06-6b9dcc25af84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:05.143296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd12e9-513d-478f-9e06-6b9dcc25af84 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:05.143939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd12e9-513d-478f-9e06-6b9dcc25af84 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:05.144598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd12e9-513d-478f-9e06-6b9dcc25af84 admin admin] PATH_INFO: `/v3/projects/cf863097bc4448e081e1b5c83eedf894/users/0471ee8d2b9345d2b86487e96b70013e/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:05.146790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd12e9-513d-478f-9e06-6b9dcc25af84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf863097bc4448e081e1b5c83eedf894, user_id=0471ee8d2b9345d2b86487e96b70013e, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:05.149389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd12e9-513d-478f-9e06-6b9dcc25af84 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:05.161631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 872/1752] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:05 2022] PUT /identity/v3/projects/cf863097bc4448e081e1b5c83eedf894/users/0471ee8d2b9345d2b86487e96b70013e/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:05.172190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bcb0a-b4cb-46b2-8564-3d4104ecac30 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:05.172787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bcb0a-b4cb-46b2-8564-3d4104ecac30 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:05.173068 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bcb0a-b4cb-46b2-8564-3d4104ecac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:05.203350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-e65bcb0a-b4cb-46b2-8564-3d4104ecac30 None None] MFA Rules not processed for user `0471ee8d2b9345d2b86487e96b70013e`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:05.204549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e65bcb0a-b4cb-46b2-8564-3d4104ecac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:05.270740 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 881/1753] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:05 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:05.287348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd12e9-513d-478f-9e06-6b9dcc25af84 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:05.349977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:05.383523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd12e9-513d-478f-9e06-6b9dcc25af84 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:05.387672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cccad-0892-4eb7-b299-44dc50223f82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:05.388289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cccad-0892-4eb7-b299-44dc50223f82 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:05.388563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cccad-0892-4eb7-b299-44dc50223f82 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:05.388829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cccad-0892-4eb7-b299-44dc50223f82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:05.398554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cccad-0892-4eb7-b299-44dc50223f82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:05.400022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cccad-0892-4eb7-b299-44dc50223f82 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:05.410558 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 873/1754] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:05 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:05.442521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bcb0a-b4cb-46b2-8564-3d4104ecac30 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:05.508075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bcb0a-b4cb-46b2-8564-3d4104ecac30 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:05.512509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:05.513132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:05.513409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:05.513681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] PATH_INFO: `/v3/users/13e3f84752dc44b79a1025c2f0ea9882` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:05.514793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e3f84752dc44b79a1025c2f0ea9882)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:05.516712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:06.086764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e3f84752dc44b79a1025c2f0ea9882'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:06.087507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e3f84752dc44b79a1025c2f0ea9882'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:06.088107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e3f84752dc44b79a1025c2f0ea9882'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:06.191694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e3f84752dc44b79a1025c2f0ea9882'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:06.205378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e3f84752dc44b79a1025c2f0ea9882'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:06.218484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 882/1755] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:05 2022] DELETE /identity/v3/users/13e3f84752dc44b79a1025c2f0ea9882 => generated 0 bytes in 776 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:06.223461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ca943-0775-4833-8d84-f4d38dc3f67c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:06.224301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ca943-0775-4833-8d84-f4d38dc3f67c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:06.224578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ca943-0775-4833-8d84-f4d38dc3f67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:06.244612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-737ca943-0775-4833-8d84-f4d38dc3f67c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:06.245812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-737ca943-0775-4833-8d84-f4d38dc3f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:06.307396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 874/1756] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:06.318128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:06.319807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:06.380645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5de9ae-3ff1-4074-8d33-ccac3d95b5a0 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:06.384257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66e053-8c42-4bd3-a631-d799fa3f7c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:06.384957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66e053-8c42-4bd3-a631-d799fa3f7c0b service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:06.385239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66e053-8c42-4bd3-a631-d799fa3f7c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:06.385512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66e053-8c42-4bd3-a631-d799fa3f7c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:06.396595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66e053-8c42-4bd3-a631-d799fa3f7c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:06.398066 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66e053-8c42-4bd3-a631-d799fa3f7c0b service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:06.408243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 883/1757] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:06.945997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:06.953077 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ca943-0775-4833-8d84-f4d38dc3f67c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:06.954894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-737ca943-0775-4833-8d84-f4d38dc3f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:06.969950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ca943-0775-4833-8d84-f4d38dc3f67c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:06.975952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:06.976715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:06.976997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:06.977274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] PATH_INFO: `/v3/projects/4f3ae1939c0247308ad975d2690c5516` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:06.978415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3ae1939c0247308ad975d2690c5516)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:06.980554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:07.142307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3ae1939c0247308ad975d2690c5516', 'request_id': 'req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:07.146705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3ae1939c0247308ad975d2690c5516', 'request_id': 'req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:07.288156 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3ae1939c0247308ad975d2690c5516', 'request_id': 'req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:07.300967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] The token cache is being invalidate because project 4f3ae1939c0247308ad975d2690c5516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:07.301264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:07.301924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:07.303467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 875/1758] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:06 2022] DELETE /identity/v3/projects/4f3ae1939c0247308ad975d2690c5516 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:07.307882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66e053-8c42-4bd3-a631-d799fa3f7c0b service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:07.318195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66e053-8c42-4bd3-a631-d799fa3f7c0b service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:07.379848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:07.380542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:07.380818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:07.381088 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] PATH_INFO: `/v3/users/0471ee8d2b9345d2b86487e96b70013e` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:07.382569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0471ee8d2b9345d2b86487e96b70013e)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:07.384879 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:07.403558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:07.592084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0471ee8d2b9345d2b86487e96b70013e'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:07.592877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0471ee8d2b9345d2b86487e96b70013e'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:07.593569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0471ee8d2b9345d2b86487e96b70013e'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:07.710875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0471ee8d2b9345d2b86487e96b70013e'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:07.723605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-7684555c-846d-4cc4-9512-ca50d2c3d44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0471ee8d2b9345d2b86487e96b70013e'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:07.743917 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 884/1759] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:07 2022] DELETE /identity/v3/users/0471ee8d2b9345d2b86487e96b70013e => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:08.255752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:08.261046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:08.274861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a2fcf4-c1fb-481d-af4c-f46e1c4e8b7c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:08.339331 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:08.339980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:08.340146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:08.340631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] PATH_INFO: `/v3/projects/cf863097bc4448e081e1b5c83eedf894` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:08.341731 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf863097bc4448e081e1b5c83eedf894)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:08.343782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:08.475010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf863097bc4448e081e1b5c83eedf894', 'request_id': 'req-b522be95-c617-461f-8c4d-557586f7a159'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:08.482667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf863097bc4448e081e1b5c83eedf894', 'request_id': 'req-b522be95-c617-461f-8c4d-557586f7a159'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:08.660030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf863097bc4448e081e1b5c83eedf894', 'request_id': 'req-b522be95-c617-461f-8c4d-557586f7a159'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:08.680611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] The token cache is being invalidate because project cf863097bc4448e081e1b5c83eedf894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:08.680905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:08.681599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b522be95-c617-461f-8c4d-557586f7a159 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:08.684809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 876/1760] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:08 2022] DELETE /identity/v3/projects/cf863097bc4448e081e1b5c83eedf894 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:08.690690 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d53b79-93f0-4658-ab15-a50ebf4d3f3e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:08.691375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d53b79-93f0-4658-ab15-a50ebf4d3f3e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:08.691375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d53b79-93f0-4658-ab15-a50ebf4d3f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:08.711353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-04d53b79-93f0-4658-ab15-a50ebf4d3f3e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:08.712579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-04d53b79-93f0-4658-ab15-a50ebf4d3f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:08.776224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 885/1761] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:08.797603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14384e-ebc5-4d1f-8b24-3796b9e1722c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:08.797895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14384e-ebc5-4d1f-8b24-3796b9e1722c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:08.798175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14384e-ebc5-4d1f-8b24-3796b9e1722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:08.820419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-0f14384e-ebc5-4d1f-8b24-3796b9e1722c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:08.821705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0f14384e-ebc5-4d1f-8b24-3796b9e1722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:08.837824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 877/1762] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:08.837917 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d53b79-93f0-4658-ab15-a50ebf4d3f3e None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:08.845246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d53b79-93f0-4658-ab15-a50ebf4d3f3e None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:08.850524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041b85e-1faa-424f-9a99-588c9b265a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:08.851195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041b85e-1faa-424f-9a99-588c9b265a24 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:08.851479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041b85e-1faa-424f-9a99-588c9b265a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:08.851762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041b85e-1faa-424f-9a99-588c9b265a24 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:08.852442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041b85e-1faa-424f-9a99-588c9b265a24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:00:08.852767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041b85e-1faa-424f-9a99-588c9b265a24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:08.854364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d041b85e-1faa-424f-9a99-588c9b265a24 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:08.865753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 886/1763] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:00:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:00:08.870160 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14384e-ebc5-4d1f-8b24-3796b9e1722c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:08.879569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14384e-ebc5-4d1f-8b24-3796b9e1722c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:08.884216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c3bc6-aa67-4fd6-babf-a3d403c41cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:08.884812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c3bc6-aa67-4fd6-babf-a3d403c41cad admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:08.885093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c3bc6-aa67-4fd6-babf-a3d403c41cad admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:08.885363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c3bc6-aa67-4fd6-babf-a3d403c41cad admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:08.885984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c3bc6-aa67-4fd6-babf-a3d403c41cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:08.887929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c3bc6-aa67-4fd6-babf-a3d403c41cad admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:09.043403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 878/1764] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:08 2022] POST /identity/v3/projects => generated 374 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:09.047766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041b85e-1faa-424f-9a99-588c9b265a24 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:09.058471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d041b85e-1faa-424f-9a99-588c9b265a24 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:09.119052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef001811-cd21-4cfa-b356-119d619b95b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:09.119667 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef001811-cd21-4cfa-b356-119d619b95b2 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:09.119948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef001811-cd21-4cfa-b356-119d619b95b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:09.120219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef001811-cd21-4cfa-b356-119d619b95b2 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:09.120855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef001811-cd21-4cfa-b356-119d619b95b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:09.122776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef001811-cd21-4cfa-b356-119d619b95b2 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:09.245832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 887/1765] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:09 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:09.249971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c3bc6-aa67-4fd6-babf-a3d403c41cad admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:09.269306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c3bc6-aa67-4fd6-babf-a3d403c41cad admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:09.282009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad499f5a-766c-4994-84e5-4f133b600469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:09.282814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499f5a-766c-4994-84e5-4f133b600469 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:09.283265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499f5a-766c-4994-84e5-4f133b600469 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:09.283765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad499f5a-766c-4994-84e5-4f133b600469 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:09.284613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad499f5a-766c-4994-84e5-4f133b600469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:09.286435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad499f5a-766c-4994-84e5-4f133b600469 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:09.299226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 879/1766] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:09 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:09.303724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef001811-cd21-4cfa-b356-119d619b95b2 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:09.317219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef001811-cd21-4cfa-b356-119d619b95b2 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:09.322835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b06a8-576f-45ac-aa80-a3c793c43e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:09.323892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b06a8-576f-45ac-aa80-a3c793c43e73 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:09.323892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b06a8-576f-45ac-aa80-a3c793c43e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:09.324255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b06a8-576f-45ac-aa80-a3c793c43e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:09.325044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b06a8-576f-45ac-aa80-a3c793c43e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:09.328966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b06a8-576f-45ac-aa80-a3c793c43e73 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:09.341885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:09.350635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 888/1767] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:09 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:00:09.354538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad499f5a-766c-4994-84e5-4f133b600469 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:09.367811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad499f5a-766c-4994-84e5-4f133b600469 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:09.375708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13418b2f-c179-4c52-b236-e00e46b2d18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:09.376504 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13418b2f-c179-4c52-b236-e00e46b2d18f admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:09.376882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13418b2f-c179-4c52-b236-e00e46b2d18f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:09.377252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13418b2f-c179-4c52-b236-e00e46b2d18f admin admin] PATH_INFO: `/v3/projects/094f376764724a21bfce5762d7961582/users/fad7917d92044f72a0d7d2b1551dfd63/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:09.394241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13418b2f-c179-4c52-b236-e00e46b2d18f admin admin] RBAC: Authorizing `identity:create_grant(project_id=094f376764724a21bfce5762d7961582, user_id=fad7917d92044f72a0d7d2b1551dfd63, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:09.396701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13418b2f-c179-4c52-b236-e00e46b2d18f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:09.534164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 880/1768] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:09 2022] PUT /identity/v3/projects/094f376764724a21bfce5762d7961582/users/fad7917d92044f72a0d7d2b1551dfd63/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:09.546069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec608b-708f-4ec6-a4a0-c62a34598527 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:09.546357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec608b-708f-4ec6-a4a0-c62a34598527 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:09.546635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec608b-708f-4ec6-a4a0-c62a34598527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:09.583454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-cfec608b-708f-4ec6-a4a0-c62a34598527 None None] MFA Rules not processed for user `fad7917d92044f72a0d7d2b1551dfd63`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:09.584765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cfec608b-708f-4ec6-a4a0-c62a34598527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:09.633102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c665a5e-e8f8-4f58-82f2-5ada7cba9c73 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:09.634003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c665a5e-e8f8-4f58-82f2-5ada7cba9c73 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:09.636667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c665a5e-e8f8-4f58-82f2-5ada7cba9c73 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:09.636667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 881/1769] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 05:00:09 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 05:00:09.650305 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09ed46-d14f-4fef-bb10-4a6d7570aba3 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:09.650618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09ed46-d14f-4fef-bb10-4a6d7570aba3 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:09.651467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09ed46-d14f-4fef-bb10-4a6d7570aba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:09.660531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 889/1770] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:09.672261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec608b-708f-4ec6-a4a0-c62a34598527 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:09.673856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cfec608b-708f-4ec6-a4a0-c62a34598527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:09.676434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9f09ed46-d14f-4fef-bb10-4a6d7570aba3 None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:09.677624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9f09ed46-d14f-4fef-bb10-4a6d7570aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:09.754410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec608b-708f-4ec6-a4a0-c62a34598527 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:09.759004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 882/1771] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 05:00:09 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:09.759487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d07dae-43d6-4c27-8db9-dde7f0df6b3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:09.760138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d07dae-43d6-4c27-8db9-dde7f0df6b3e service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:09.760419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d07dae-43d6-4c27-8db9-dde7f0df6b3e service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:09.760696 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d07dae-43d6-4c27-8db9-dde7f0df6b3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:09.769817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d07dae-43d6-4c27-8db9-dde7f0df6b3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:09.771275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d07dae-43d6-4c27-8db9-dde7f0df6b3e service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:09.773771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09ed46-d14f-4fef-bb10-4a6d7570aba3 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:09.775428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9f09ed46-d14f-4fef-bb10-4a6d7570aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:09.781441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 890/1772] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:00:09.792724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09ed46-d14f-4fef-bb10-4a6d7570aba3 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:09.800051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319517ca-41b5-4bff-a229-2b2dcaa6a8c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:09.800750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319517ca-41b5-4bff-a229-2b2dcaa6a8c8 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:09.801022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319517ca-41b5-4bff-a229-2b2dcaa6a8c8 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:09.814620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319517ca-41b5-4bff-a229-2b2dcaa6a8c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:09.814620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319517ca-41b5-4bff-a229-2b2dcaa6a8c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:09.814620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319517ca-41b5-4bff-a229-2b2dcaa6a8c8 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:09.818666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d07dae-43d6-4c27-8db9-dde7f0df6b3e service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:09.824600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 883/1773] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:09 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:00:09.841884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:09.896608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d07dae-43d6-4c27-8db9-dde7f0df6b3e service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:09.902220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:09.902982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:09.903520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:09.903914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] PATH_INFO: `/v3/users/fad7917d92044f72a0d7d2b1551dfd63` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:09.905475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=fad7917d92044f72a0d7d2b1551dfd63)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:09.907678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:10.095688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fad7917d92044f72a0d7d2b1551dfd63'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:10.096872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fad7917d92044f72a0d7d2b1551dfd63'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:10.097555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fad7917d92044f72a0d7d2b1551dfd63'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:10.179151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fad7917d92044f72a0d7d2b1551dfd63'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:10.192210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fad7917d92044f72a0d7d2b1551dfd63'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:10.204516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 891/1774] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:09 2022] DELETE /identity/v3/users/fad7917d92044f72a0d7d2b1551dfd63 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:10.209867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff9116-14d8-44f6-a29f-cd4c9959b8d7 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:10.210154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff9116-14d8-44f6-a29f-cd4c9959b8d7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:10.210431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff9116-14d8-44f6-a29f-cd4c9959b8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:10.229504 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-1cff9116-14d8-44f6-a29f-cd4c9959b8d7 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:10.230704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-1cff9116-14d8-44f6-a29f-cd4c9959b8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:10.289749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 884/1775] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:10.300069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:10.301464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:10.358898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a72ae-a2ab-40e4-91df-f58fdb012fab admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:10.363063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a282cef2-8db5-48c1-b153-dcc8f6c110fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:10.363726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282cef2-8db5-48c1-b153-dcc8f6c110fe service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:10.364000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282cef2-8db5-48c1-b153-dcc8f6c110fe service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:10.364268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282cef2-8db5-48c1-b153-dcc8f6c110fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:10.373035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282cef2-8db5-48c1-b153-dcc8f6c110fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:10.374446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a282cef2-8db5-48c1-b153-dcc8f6c110fe service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:10.384729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 892/1776] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:10.966067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff9116-14d8-44f6-a29f-cd4c9959b8d7 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:10.966067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-1cff9116-14d8-44f6-a29f-cd4c9959b8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:10.979405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff9116-14d8-44f6-a29f-cd4c9959b8d7 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:10.985106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:10.985743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:10.986021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:10.986296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] PATH_INFO: `/v3/projects/094f376764724a21bfce5762d7961582` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:10.988133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=094f376764724a21bfce5762d7961582)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:10.990666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:11.157052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094f376764724a21bfce5762d7961582', 'request_id': 'req-0b932890-eec8-4179-b395-ac5e72385d32'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:11.163877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094f376764724a21bfce5762d7961582', 'request_id': 'req-0b932890-eec8-4179-b395-ac5e72385d32'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:11.317065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094f376764724a21bfce5762d7961582', 'request_id': 'req-0b932890-eec8-4179-b395-ac5e72385d32'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:11.328285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] The token cache is being invalidate because project 094f376764724a21bfce5762d7961582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:11.328571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:11.329195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0b932890-eec8-4179-b395-ac5e72385d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:11.331529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 885/1777] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:10 2022] DELETE /identity/v3/projects/094f376764724a21bfce5762d7961582 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:11.336417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b912a9-ae2e-4e32-ae85-894e892262ec None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:11.336867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b912a9-ae2e-4e32-ae85-894e892262ec None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:11.336959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b912a9-ae2e-4e32-ae85-894e892262ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:11.356509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-73b912a9-ae2e-4e32-ae85-894e892262ec None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:11.357697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-73b912a9-ae2e-4e32-ae85-894e892262ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:11.415704 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 893/1778] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:11.423399 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d6839-dbe1-480c-93c7-6e2a9db9037e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:11.423681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d6839-dbe1-480c-93c7-6e2a9db9037e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:11.423950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d6839-dbe1-480c-93c7-6e2a9db9037e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:11.444524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-984d6839-dbe1-480c-93c7-6e2a9db9037e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:11.445806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-984d6839-dbe1-480c-93c7-6e2a9db9037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:11.454016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 886/1779] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:11.458967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b912a9-ae2e-4e32-ae85-894e892262ec None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:11.475358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b912a9-ae2e-4e32-ae85-894e892262ec None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:11.480119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbeb00-3b96-4a67-83e0-e542e5d0b0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:11.480727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbeb00-3b96-4a67-83e0-e542e5d0b0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:11.481010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbeb00-3b96-4a67-83e0-e542e5d0b0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:11.481283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbeb00-3b96-4a67-83e0-e542e5d0b0f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:11.481929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbeb00-3b96-4a67-83e0-e542e5d0b0f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:00:11.482228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbeb00-3b96-4a67-83e0-e542e5d0b0f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:11.483851 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbeb00-3b96-4a67-83e0-e542e5d0b0f4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:11.495785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 894/1780] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:00:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:00:11.500295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d6839-dbe1-480c-93c7-6e2a9db9037e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:11.510155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d6839-dbe1-480c-93c7-6e2a9db9037e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:11.514944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d3286-9531-4e4d-a261-e34eca87861f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:11.515591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d3286-9531-4e4d-a261-e34eca87861f admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:11.515865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d3286-9531-4e4d-a261-e34eca87861f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:11.516133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d3286-9531-4e4d-a261-e34eca87861f admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:11.516762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d3286-9531-4e4d-a261-e34eca87861f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:11.519274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d3286-9531-4e4d-a261-e34eca87861f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:11.638210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 887/1781] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:11 2022] POST /identity/v3/projects => generated 376 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:11.642476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbeb00-3b96-4a67-83e0-e542e5d0b0f4 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:11.654005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbeb00-3b96-4a67-83e0-e542e5d0b0f4 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:11.713128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc9a97-e6ee-4f58-83d2-9e4eb9281440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:11.713736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc9a97-e6ee-4f58-83d2-9e4eb9281440 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:11.714012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc9a97-e6ee-4f58-83d2-9e4eb9281440 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:11.714273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc9a97-e6ee-4f58-83d2-9e4eb9281440 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:11.714985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc9a97-e6ee-4f58-83d2-9e4eb9281440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:11.716939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc9a97-e6ee-4f58-83d2-9e4eb9281440 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:11.756516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:11.846083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 895/1782] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:11 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:00:11.847662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d3286-9531-4e4d-a261-e34eca87861f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:11.864675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d3286-9531-4e4d-a261-e34eca87861f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:11.877150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3745d-a355-4989-a0f5-cfe8b5a25066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:11.877929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3745d-a355-4989-a0f5-cfe8b5a25066 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:11.878200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3745d-a355-4989-a0f5-cfe8b5a25066 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:11.878466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3745d-a355-4989-a0f5-cfe8b5a25066 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:11.879653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3745d-a355-4989-a0f5-cfe8b5a25066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:11.882107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3745d-a355-4989-a0f5-cfe8b5a25066 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:11.895761 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 888/1783] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:11 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:11.902072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc9a97-e6ee-4f58-83d2-9e4eb9281440 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:11.913117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc9a97-e6ee-4f58-83d2-9e4eb9281440 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:11.917869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117259a-d33e-441a-a6a1-f28fdbefb297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:11.918477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117259a-d33e-441a-a6a1-f28fdbefb297 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:11.918754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117259a-d33e-441a-a6a1-f28fdbefb297 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:11.919045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117259a-d33e-441a-a6a1-f28fdbefb297 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:11.919730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117259a-d33e-441a-a6a1-f28fdbefb297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:11.921351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117259a-d33e-441a-a6a1-f28fdbefb297 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:11.933440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 896/1784] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:11 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:11.938810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3745d-a355-4989-a0f5-cfe8b5a25066 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:11.949444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3745d-a355-4989-a0f5-cfe8b5a25066 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:11.953954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186f075-528f-47ce-819f-b2b6851da715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:11.954602 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186f075-528f-47ce-819f-b2b6851da715 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:11.954879 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186f075-528f-47ce-819f-b2b6851da715 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:11.955198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5186f075-528f-47ce-819f-b2b6851da715 admin admin] PATH_INFO: `/v3/projects/7b3d1c2e23a64b849d240f8a7ce9dcc0/users/33e7a831c55e41e3bf6aa0921c92ce87/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:11.969804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186f075-528f-47ce-819f-b2b6851da715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3d1c2e23a64b849d240f8a7ce9dcc0, user_id=33e7a831c55e41e3bf6aa0921c92ce87, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:11.972588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5186f075-528f-47ce-819f-b2b6851da715 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:11.985265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:12.048170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 889/1785] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:11 2022] PUT /identity/v3/projects/7b3d1c2e23a64b849d240f8a7ce9dcc0/users/33e7a831c55e41e3bf6aa0921c92ce87/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:12.059754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf8115-99ad-41dd-aa32-5bd29a8315a7 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:12.069146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf8115-99ad-41dd-aa32-5bd29a8315a7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:12.069146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf8115-99ad-41dd-aa32-5bd29a8315a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:12.087570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-19bf8115-99ad-41dd-aa32-5bd29a8315a7 None None] MFA Rules not processed for user `33e7a831c55e41e3bf6aa0921c92ce87`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:12.088836 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-19bf8115-99ad-41dd-aa32-5bd29a8315a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:12.154302 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 897/1786] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:12.165703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186f075-528f-47ce-819f-b2b6851da715 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:12.167445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5186f075-528f-47ce-819f-b2b6851da715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:12.224424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5186f075-528f-47ce-819f-b2b6851da715 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:12.229872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82ea38-9baf-47fb-8fdd-aee4161993f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:12.229872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82ea38-9baf-47fb-8fdd-aee4161993f2 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:12.229872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82ea38-9baf-47fb-8fdd-aee4161993f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:12.229872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82ea38-9baf-47fb-8fdd-aee4161993f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:12.237500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82ea38-9baf-47fb-8fdd-aee4161993f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:12.238905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82ea38-9baf-47fb-8fdd-aee4161993f2 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:12.247475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 890/1787] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:12.248151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:12.333851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:13.248202 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:13.374140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:14.133429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:14.196492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:14.670636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:14.807893 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:14.978488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf8115-99ad-41dd-aa32-5bd29a8315a7 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:14.978488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-19bf8115-99ad-41dd-aa32-5bd29a8315a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:14.997176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf8115-99ad-41dd-aa32-5bd29a8315a7 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:15.012656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c462ce-2763-4dda-a1c5-61d9c3a1f7ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:15.013280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c462ce-2763-4dda-a1c5-61d9c3a1f7ec service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:15.013564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c462ce-2763-4dda-a1c5-61d9c3a1f7ec service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:15.013836 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c462ce-2763-4dda-a1c5-61d9c3a1f7ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:15.015657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c8c462ce-2763-4dda-a1c5-61d9c3a1f7ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:15.078272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c462ce-2763-4dda-a1c5-61d9c3a1f7ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:15.079763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c462ce-2763-4dda-a1c5-61d9c3a1f7ec service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:15.089171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 898/1788] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:14 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:22.692031 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82ea38-9baf-47fb-8fdd-aee4161993f2 service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:22.693579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-dd82ea38-9baf-47fb-8fdd-aee4161993f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:22.757567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82ea38-9baf-47fb-8fdd-aee4161993f2 service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:22.762360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:22.766857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:22.766857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:22.766857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] PATH_INFO: `/v3/users/7d1603851b7d4baabf48b3fac47433de` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:22.766857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d1603851b7d4baabf48b3fac47433de)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:22.766857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:23.044010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d1603851b7d4baabf48b3fac47433de'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:23.045385 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d1603851b7d4baabf48b3fac47433de'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:23.045385 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d1603851b7d4baabf48b3fac47433de'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:23.172495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d1603851b7d4baabf48b3fac47433de'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:23.186186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d1603851b7d4baabf48b3fac47433de'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:23.198702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 891/1789] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:22 2022] DELETE /identity/v3/users/7d1603851b7d4baabf48b3fac47433de => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:23.203183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ff5cb-509e-4359-8350-e4b88fd7874d None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:23.203680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ff5cb-509e-4359-8350-e4b88fd7874d None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:23.203775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ff5cb-509e-4359-8350-e4b88fd7874d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:23.222926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-379ff5cb-509e-4359-8350-e4b88fd7874d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:23.224151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-379ff5cb-509e-4359-8350-e4b88fd7874d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:23.232336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:23.282553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 899/1790] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:23.294245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:23.295048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:23.355296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170553b-3107-40b0-b321-92aa1cdc4242 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:23.359122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab20f18-e713-4446-997b-1023fe77118d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:23.359906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab20f18-e713-4446-997b-1023fe77118d service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:23.360186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab20f18-e713-4446-997b-1023fe77118d service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:23.360458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab20f18-e713-4446-997b-1023fe77118d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:23.369698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab20f18-e713-4446-997b-1023fe77118d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:23.371187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab20f18-e713-4446-997b-1023fe77118d service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:23.380690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 892/1791] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:23.849716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ff5cb-509e-4359-8350-e4b88fd7874d None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:23.849716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-379ff5cb-509e-4359-8350-e4b88fd7874d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:23.860833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ff5cb-509e-4359-8350-e4b88fd7874d None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:23.865721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:23.866360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:23.866639 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:23.866909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] PATH_INFO: `/v3/projects/dca24fb48ad842e6b12efd85726dd82a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:23.868099 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dca24fb48ad842e6b12efd85726dd82a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:23.870053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:24.049995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dca24fb48ad842e6b12efd85726dd82a', 'request_id': 'req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:24.064410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dca24fb48ad842e6b12efd85726dd82a', 'request_id': 'req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:24.072974 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dca24fb48ad842e6b12efd85726dd82a', 'request_id': 'req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:24.154868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] The token cache is being invalidate because project dca24fb48ad842e6b12efd85726dd82a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:24.155421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:24.156157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:24.157668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 900/1792] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:23 2022] DELETE /identity/v3/projects/dca24fb48ad842e6b12efd85726dd82a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:24.161774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab20f18-e713-4446-997b-1023fe77118d service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:24.166729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:24.172868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab20f18-e713-4446-997b-1023fe77118d service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:24.238701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:24.240440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:24.241669 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:24.242114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] PATH_INFO: `/v3/users/786077efd1fa4e598d53a905dadcb6cb` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:24.247777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=786077efd1fa4e598d53a905dadcb6cb)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:24.252613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:24.489420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '786077efd1fa4e598d53a905dadcb6cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:24.490060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '786077efd1fa4e598d53a905dadcb6cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:24.490660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '786077efd1fa4e598d53a905dadcb6cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:24.566228 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '786077efd1fa4e598d53a905dadcb6cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:24.578794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '786077efd1fa4e598d53a905dadcb6cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:24.590909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 893/1793] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:24 2022] DELETE /identity/v3/users/786077efd1fa4e598d53a905dadcb6cb => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:25.031251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:25.032769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:25.047213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2899d6b-57e6-4434-9adf-dd6c8fed14e3 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:25.111059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:25.111793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:25.112137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:25.112476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] PATH_INFO: `/v3/projects/0548b560bb5f45a69d87fe348558a505` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:25.113815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0548b560bb5f45a69d87fe348558a505)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:25.116054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:25.253576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:25.260418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0548b560bb5f45a69d87fe348558a505', 'request_id': 'req-eb17d8a2-5c2f-492c-a5ed-250d66efe680'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:25.266749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0548b560bb5f45a69d87fe348558a505', 'request_id': 'req-eb17d8a2-5c2f-492c-a5ed-250d66efe680'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:25.273090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0548b560bb5f45a69d87fe348558a505', 'request_id': 'req-eb17d8a2-5c2f-492c-a5ed-250d66efe680'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:25.372016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] The token cache is being invalidate because project 0548b560bb5f45a69d87fe348558a505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:25.373345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:25.373521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:25.375134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 901/1794] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:25 2022] DELETE /identity/v3/projects/0548b560bb5f45a69d87fe348558a505 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:25.379961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:25.381394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:25.395531 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7d32a-df01-4f54-b723-a1400f825450 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:25.473187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:25.474053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:25.474729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:25.475419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] PATH_INFO: `/v3/users/1c67e25381974886a89d25b7a420b3c5` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:25.476604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c67e25381974886a89d25b7a420b3c5)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:25.478538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:25.504578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:25.741064 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c67e25381974886a89d25b7a420b3c5'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:25.741726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c67e25381974886a89d25b7a420b3c5'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:25.742398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c67e25381974886a89d25b7a420b3c5'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:25.857831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c67e25381974886a89d25b7a420b3c5'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:25.870597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-198dea4f-50eb-45d1-8688-38db0f1e538e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c67e25381974886a89d25b7a420b3c5'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:25.883393 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 894/1795] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:25 2022] DELETE /identity/v3/users/1c67e25381974886a89d25b7a420b3c5 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:26.310712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:26.312702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:26.325567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17d8a2-5c2f-492c-a5ed-250d66efe680 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:26.402395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:26.403222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:26.403612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:26.404011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] PATH_INFO: `/v3/projects/46d018ad4bda44268f69c9c065e96366` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:26.405389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=46d018ad4bda44268f69c9c065e96366)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:26.408307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:26.566005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46d018ad4bda44268f69c9c065e96366', 'request_id': 'req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:26.572797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46d018ad4bda44268f69c9c065e96366', 'request_id': 'req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:26.579870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46d018ad4bda44268f69c9c065e96366', 'request_id': 'req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:26.763310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] The token cache is being invalidate because project 46d018ad4bda44268f69c9c065e96366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:26.763603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:26.764265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-da3bbdc9-8aa5-4ff1-970f-14c785b2e3bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:26.766254 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 902/1796] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:26 2022] DELETE /identity/v3/projects/46d018ad4bda44268f69c9c065e96366 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:26.772742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df7829-01d0-441b-8e6f-d3457c1b96df None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:26.775699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df7829-01d0-441b-8e6f-d3457c1b96df None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:26.775699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df7829-01d0-441b-8e6f-d3457c1b96df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:26.793963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-14df7829-01d0-441b-8e6f-d3457c1b96df None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:26.795198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-14df7829-01d0-441b-8e6f-d3457c1b96df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:26.856494 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 895/1797] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:26.865413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14368-e74a-4631-a1e6-aad60d935c00 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:26.865697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14368-e74a-4631-a1e6-aad60d935c00 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:26.865971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a14368-e74a-4631-a1e6-aad60d935c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:26.894389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-86a14368-e74a-4631-a1e6-aad60d935c00 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:26.898232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-86a14368-e74a-4631-a1e6-aad60d935c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:26.915233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 903/1798] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:26.924394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df7829-01d0-441b-8e6f-d3457c1b96df None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:26.938652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df7829-01d0-441b-8e6f-d3457c1b96df None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:26.948654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d5932-f3ab-4941-a5ed-15b51a4595ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:26.949291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d5932-f3ab-4941-a5ed-15b51a4595ad admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:26.949571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d5932-f3ab-4941-a5ed-15b51a4595ad admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:26.949844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d5932-f3ab-4941-a5ed-15b51a4595ad admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:26.950598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d5932-f3ab-4941-a5ed-15b51a4595ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:00:26.950858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d5932-f3ab-4941-a5ed-15b51a4595ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:26.952543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d5932-f3ab-4941-a5ed-15b51a4595ad admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:26.967057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 896/1799] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:00:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:00:26.976764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a14368-e74a-4631-a1e6-aad60d935c00 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:26.979857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a14368-e74a-4631-a1e6-aad60d935c00 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:26.984688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74443f-dcdc-4d6e-b82e-f5f039ebfb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:26.985298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74443f-dcdc-4d6e-b82e-f5f039ebfb57 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:26.985581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74443f-dcdc-4d6e-b82e-f5f039ebfb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:26.985854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74443f-dcdc-4d6e-b82e-f5f039ebfb57 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:26.986492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74443f-dcdc-4d6e-b82e-f5f039ebfb57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:26.989128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74443f-dcdc-4d6e-b82e-f5f039ebfb57 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:27.125227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 904/1800] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:26 2022] POST /identity/v3/projects => generated 380 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:27.126264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d5932-f3ab-4941-a5ed-15b51a4595ad admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:27.137268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d5932-f3ab-4941-a5ed-15b51a4595ad admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:27.222822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4a15e-0a59-4c96-a398-0d098f8471f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:27.223458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4a15e-0a59-4c96-a398-0d098f8471f1 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:27.223731 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4a15e-0a59-4c96-a398-0d098f8471f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:27.223999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4a15e-0a59-4c96-a398-0d098f8471f1 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:27.224626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4a15e-0a59-4c96-a398-0d098f8471f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:27.226510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4a15e-0a59-4c96-a398-0d098f8471f1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:27.322126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 897/1801] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:27 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:27.323443 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74443f-dcdc-4d6e-b82e-f5f039ebfb57 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:27.334959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74443f-dcdc-4d6e-b82e-f5f039ebfb57 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:27.351525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86dd348-9fde-4d8a-8f9c-7bd9bf95a010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:27.351525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dd348-9fde-4d8a-8f9c-7bd9bf95a010 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:27.351525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dd348-9fde-4d8a-8f9c-7bd9bf95a010 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:27.351525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86dd348-9fde-4d8a-8f9c-7bd9bf95a010 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:27.351945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86dd348-9fde-4d8a-8f9c-7bd9bf95a010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:27.351945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86dd348-9fde-4d8a-8f9c-7bd9bf95a010 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:27.358197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 905/1802] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:27 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:27.359522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4a15e-0a59-4c96-a398-0d098f8471f1 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:27.370487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4a15e-0a59-4c96-a398-0d098f8471f1 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:27.375821 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93119377-aad4-4016-996f-e3857ff09656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:27.376165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93119377-aad4-4016-996f-e3857ff09656 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:27.376448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93119377-aad4-4016-996f-e3857ff09656 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:27.376723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93119377-aad4-4016-996f-e3857ff09656 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:27.377394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93119377-aad4-4016-996f-e3857ff09656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:27.379115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93119377-aad4-4016-996f-e3857ff09656 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:27.390494 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 898/1803] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:27 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:27.394974 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86dd348-9fde-4d8a-8f9c-7bd9bf95a010 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:27.405095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86dd348-9fde-4d8a-8f9c-7bd9bf95a010 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:27.410414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dd09f-4032-462b-bba1-9708177f1258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:27.411085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dd09f-4032-462b-bba1-9708177f1258 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:27.411363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dd09f-4032-462b-bba1-9708177f1258 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:27.411635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625dd09f-4032-462b-bba1-9708177f1258 admin admin] PATH_INFO: `/v3/projects/86e39994e8db4f128d5a6183b8d91cd6/users/773b9b9aaec643eeba397e05fd0cb178/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:27.426633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dd09f-4032-462b-bba1-9708177f1258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86e39994e8db4f128d5a6183b8d91cd6, user_id=773b9b9aaec643eeba397e05fd0cb178, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:27.429546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625dd09f-4032-462b-bba1-9708177f1258 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:27.431871 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:27.573661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 906/1804] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:27 2022] PUT /identity/v3/projects/86e39994e8db4f128d5a6183b8d91cd6/users/773b9b9aaec643eeba397e05fd0cb178/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:27.586182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14584ed9-366e-4b95-a0dd-fa46736b1168 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:27.587268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14584ed9-366e-4b95-a0dd-fa46736b1168 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:27.587268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14584ed9-366e-4b95-a0dd-fa46736b1168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:27.601119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:27.620693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-14584ed9-366e-4b95-a0dd-fa46736b1168 None None] MFA Rules not processed for user `773b9b9aaec643eeba397e05fd0cb178`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:27.621956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-14584ed9-366e-4b95-a0dd-fa46736b1168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:27.688295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 899/1805] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:00:27.692988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dd09f-4032-462b-bba1-9708177f1258 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:27.762059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625dd09f-4032-462b-bba1-9708177f1258 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:27.766713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5560e-1a22-48a8-8143-f55cd648e073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:27.767345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5560e-1a22-48a8-8143-f55cd648e073 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:27.767666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5560e-1a22-48a8-8143-f55cd648e073 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:27.767944 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5560e-1a22-48a8-8143-f55cd648e073 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:27.769186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5560e-1a22-48a8-8143-f55cd648e073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:27.771093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5560e-1a22-48a8-8143-f55cd648e073 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:27.847745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 907/1806] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:27 2022] POST /identity/v3/projects => generated 380 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:00:27.851769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14584ed9-366e-4b95-a0dd-fa46736b1168 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:27.862693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14584ed9-366e-4b95-a0dd-fa46736b1168 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:27.921449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad7760-344d-4be1-abee-43948e0338f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:27.922094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad7760-344d-4be1-abee-43948e0338f3 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:27.922369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad7760-344d-4be1-abee-43948e0338f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:27.922654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ad7760-344d-4be1-abee-43948e0338f3 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:27.924904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad7760-344d-4be1-abee-43948e0338f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:27.927540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ad7760-344d-4be1-abee-43948e0338f3 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:27.984614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 900/1807] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:27 2022] POST /identity/v3/users => generated 291 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:27.989067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5560e-1a22-48a8-8143-f55cd648e073 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:27.999900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5560e-1a22-48a8-8143-f55cd648e073 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:28.004938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449453e1-07ee-406c-a403-1b33a7389604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:28.005552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449453e1-07ee-406c-a403-1b33a7389604 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:28.005879 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449453e1-07ee-406c-a403-1b33a7389604 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:28.006153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449453e1-07ee-406c-a403-1b33a7389604 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:28.006791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449453e1-07ee-406c-a403-1b33a7389604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:28.008614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449453e1-07ee-406c-a403-1b33a7389604 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:28.021802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 908/1808] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:27 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:28.025945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad7760-344d-4be1-abee-43948e0338f3 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:28.036007 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ad7760-344d-4be1-abee-43948e0338f3 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:28.040948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a3d53-f717-4ee2-adea-8879c2a75427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:28.041582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a3d53-f717-4ee2-adea-8879c2a75427 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:28.041856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a3d53-f717-4ee2-adea-8879c2a75427 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:28.042175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a3d53-f717-4ee2-adea-8879c2a75427 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:28.042813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a3d53-f717-4ee2-adea-8879c2a75427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:28.044704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a3d53-f717-4ee2-adea-8879c2a75427 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:28.056139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 901/1809] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:28 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:28.060304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449453e1-07ee-406c-a403-1b33a7389604 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:28.072236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449453e1-07ee-406c-a403-1b33a7389604 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:28.081922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aac738-fce6-4a08-ac04-c7581138f069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:28.082821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac738-fce6-4a08-ac04-c7581138f069 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:28.083469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac738-fce6-4a08-ac04-c7581138f069 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:28.083932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac738-fce6-4a08-ac04-c7581138f069 admin admin] PATH_INFO: `/v3/projects/e962c399b4c14a72b5e51c6a9cb1e148/users/c55d94c8b87e4eb69a8ab330d3f9e1f3/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:28.102373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aac738-fce6-4a08-ac04-c7581138f069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e962c399b4c14a72b5e51c6a9cb1e148, user_id=c55d94c8b87e4eb69a8ab330d3f9e1f3, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:28.105133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aac738-fce6-4a08-ac04-c7581138f069 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:28.200300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 909/1810] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:28 2022] PUT /identity/v3/projects/e962c399b4c14a72b5e51c6a9cb1e148/users/c55d94c8b87e4eb69a8ab330d3f9e1f3/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:28.210877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1bf94-83a5-4c7a-9b8c-023b6c13c175 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:28.211190 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1bf94-83a5-4c7a-9b8c-023b6c13c175 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:28.211465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1bf94-83a5-4c7a-9b8c-023b6c13c175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:28.240924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-92d1bf94-83a5-4c7a-9b8c-023b6c13c175 None None] MFA Rules not processed for user `c55d94c8b87e4eb69a8ab330d3f9e1f3`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:28.242155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-92d1bf94-83a5-4c7a-9b8c-023b6c13c175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:28.307625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 902/1811] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:28.312054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aac738-fce6-4a08-ac04-c7581138f069 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:28.375273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aac738-fce6-4a08-ac04-c7581138f069 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:28.382684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fbfa2-b6ea-451d-b057-b76deebd4e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:28.383405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fbfa2-b6ea-451d-b057-b76deebd4e47 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:28.383613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fbfa2-b6ea-451d-b057-b76deebd4e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:28.383888 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fbfa2-b6ea-451d-b057-b76deebd4e47 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:28.384514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fbfa2-b6ea-451d-b057-b76deebd4e47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:28.386413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fbfa2-b6ea-451d-b057-b76deebd4e47 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:28.503125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 910/1812] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:28 2022] POST /identity/v3/projects => generated 380 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:28.507794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1bf94-83a5-4c7a-9b8c-023b6c13c175 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:28.515126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:28.519577 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1bf94-83a5-4c7a-9b8c-023b6c13c175 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:28.575910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b8faf-4d4f-46d8-b63f-6971c5a21f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:28.576704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b8faf-4d4f-46d8-b63f-6971c5a21f60 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:28.581928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b8faf-4d4f-46d8-b63f-6971c5a21f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:28.581928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b8faf-4d4f-46d8-b63f-6971c5a21f60 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:28.581928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b8faf-4d4f-46d8-b63f-6971c5a21f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:28.581928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b8faf-4d4f-46d8-b63f-6971c5a21f60 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:28.778276 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 903/1813] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:28 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:28.782221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fbfa2-b6ea-451d-b057-b76deebd4e47 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:28.794413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fbfa2-b6ea-451d-b057-b76deebd4e47 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:28.799805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699bdeb-3190-4060-ba8a-36a0031d4169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:28.800345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699bdeb-3190-4060-ba8a-36a0031d4169 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:28.800779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699bdeb-3190-4060-ba8a-36a0031d4169 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:28.801053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699bdeb-3190-4060-ba8a-36a0031d4169 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:28.801701 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699bdeb-3190-4060-ba8a-36a0031d4169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:28.803565 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699bdeb-3190-4060-ba8a-36a0031d4169 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:28.822286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b8faf-4d4f-46d8-b63f-6971c5a21f60 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:28.823034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 911/1814] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:28 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:28.834932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b8faf-4d4f-46d8-b63f-6971c5a21f60 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:28.839474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e98b8f-3e27-487e-a93a-05ffa2bba630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:28.842238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e98b8f-3e27-487e-a93a-05ffa2bba630 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:28.842238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e98b8f-3e27-487e-a93a-05ffa2bba630 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:28.842238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e98b8f-3e27-487e-a93a-05ffa2bba630 admin admin] PATH_INFO: `/v3/projects/c664152846de4817a0c767c1de7960e1/users/352865e49b7b41df8a8fa0bf9d2c5359/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:28.859662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e98b8f-3e27-487e-a93a-05ffa2bba630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c664152846de4817a0c767c1de7960e1, user_id=352865e49b7b41df8a8fa0bf9d2c5359, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:28.860236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e98b8f-3e27-487e-a93a-05ffa2bba630 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:28.946172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 904/1815] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:28 2022] PUT /identity/v3/projects/c664152846de4817a0c767c1de7960e1/users/352865e49b7b41df8a8fa0bf9d2c5359/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:28.950362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699bdeb-3190-4060-ba8a-36a0031d4169 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:28.961745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699bdeb-3190-4060-ba8a-36a0031d4169 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:28.978636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:29.022728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbab43b-b73e-478a-b7b4-d8b8fdce3dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:29.023365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbab43b-b73e-478a-b7b4-d8b8fdce3dae admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:29.023644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbab43b-b73e-478a-b7b4-d8b8fdce3dae admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:29.023976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbab43b-b73e-478a-b7b4-d8b8fdce3dae admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:29.024613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbab43b-b73e-478a-b7b4-d8b8fdce3dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:29.026218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbab43b-b73e-478a-b7b4-d8b8fdce3dae admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:29.035910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 912/1816] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:28 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:29.040103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e98b8f-3e27-487e-a93a-05ffa2bba630 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:29.050199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e98b8f-3e27-487e-a93a-05ffa2bba630 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:29.055221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3df6c3-d759-4e12-8bf2-759577afeec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:29.055869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3df6c3-d759-4e12-8bf2-759577afeec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:29.056143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3df6c3-d759-4e12-8bf2-759577afeec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:29.056409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3df6c3-d759-4e12-8bf2-759577afeec7 admin admin] PATH_INFO: `/v3/projects/c664152846de4817a0c767c1de7960e1/users/352865e49b7b41df8a8fa0bf9d2c5359/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:29.058319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3df6c3-d759-4e12-8bf2-759577afeec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c664152846de4817a0c767c1de7960e1, user_id=352865e49b7b41df8a8fa0bf9d2c5359, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:29.060984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3df6c3-d759-4e12-8bf2-759577afeec7 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:29.072549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 905/1817] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:29 2022] PUT /identity/v3/projects/c664152846de4817a0c767c1de7960e1/users/352865e49b7b41df8a8fa0bf9d2c5359/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:29.083464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db32dd-8856-4698-8764-98c8e183e84c None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:29.083753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db32dd-8856-4698-8764-98c8e183e84c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:29.084029 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db32dd-8856-4698-8764-98c8e183e84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:29.111784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c4db32dd-8856-4698-8764-98c8e183e84c None None] MFA Rules not processed for user `352865e49b7b41df8a8fa0bf9d2c5359`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:29.112998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c4db32dd-8856-4698-8764-98c8e183e84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:29.222643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 913/1818] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:29 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:29.240255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3df6c3-d759-4e12-8bf2-759577afeec7 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:29.242206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-dc3df6c3-d759-4e12-8bf2-759577afeec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:29.318318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3df6c3-d759-4e12-8bf2-759577afeec7 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:29.323464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a683c4f-b2a4-4d06-b70f-5029369628fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:29.323464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a683c4f-b2a4-4d06-b70f-5029369628fb service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:29.323464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a683c4f-b2a4-4d06-b70f-5029369628fb service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:29.323464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a683c4f-b2a4-4d06-b70f-5029369628fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:29.334380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a683c4f-b2a4-4d06-b70f-5029369628fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:29.335610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a683c4f-b2a4-4d06-b70f-5029369628fb service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:29.355166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 906/1819] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:29 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:29.463941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db32dd-8856-4698-8764-98c8e183e84c None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:29.465498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c4db32dd-8856-4698-8764-98c8e183e84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:29.479282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db32dd-8856-4698-8764-98c8e183e84c None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:29.484317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62446551-025b-4bb0-8b8b-cba0bf23b2cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:29.484317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62446551-025b-4bb0-8b8b-cba0bf23b2cb service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:29.484671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62446551-025b-4bb0-8b8b-cba0bf23b2cb service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:29.484890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62446551-025b-4bb0-8b8b-cba0bf23b2cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:29.547531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62446551-025b-4bb0-8b8b-cba0bf23b2cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:29.548879 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62446551-025b-4bb0-8b8b-cba0bf23b2cb service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:29.559564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 914/1820] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:36.155151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a683c4f-b2a4-4d06-b70f-5029369628fb service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:36.280724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a683c4f-b2a4-4d06-b70f-5029369628fb service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:36.289380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:36.290535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:36.291092 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:36.291443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] PATH_INFO: `/v3/users/773b9b9aaec643eeba397e05fd0cb178` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:36.293152 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=773b9b9aaec643eeba397e05fd0cb178)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:36.300713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:36.300713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:36.711573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '773b9b9aaec643eeba397e05fd0cb178'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:36.712473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '773b9b9aaec643eeba397e05fd0cb178'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:36.714561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '773b9b9aaec643eeba397e05fd0cb178'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:36.850590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '773b9b9aaec643eeba397e05fd0cb178'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:36.863476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '773b9b9aaec643eeba397e05fd0cb178'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:36.876389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 907/1821] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:36 2022] DELETE /identity/v3/users/773b9b9aaec643eeba397e05fd0cb178 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:36.880980 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c8d66-b63e-429b-9968-b77b71cff497 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:36.881463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c8d66-b63e-429b-9968-b77b71cff497 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:36.881647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c8d66-b63e-429b-9968-b77b71cff497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:36.901620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-935c8d66-b63e-429b-9968-b77b71cff497 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:36.902775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-935c8d66-b63e-429b-9968-b77b71cff497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:36.943394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:36.972437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 915/1822] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:36.981987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:36.983490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:37.046486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a612b41-b819-47a4-ae36-3151cf8e04f1 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:37.051157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ed7f3-c723-4c7d-ba22-93e73559a5f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:37.051770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ed7f3-c723-4c7d-ba22-93e73559a5f0 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:37.052048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ed7f3-c723-4c7d-ba22-93e73559a5f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:37.052316 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ed7f3-c723-4c7d-ba22-93e73559a5f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:37.067414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ed7f3-c723-4c7d-ba22-93e73559a5f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:37.068856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ed7f3-c723-4c7d-ba22-93e73559a5f0 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:37.078754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 908/1823] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:37.630032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c8d66-b63e-429b-9968-b77b71cff497 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:37.631707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-935c8d66-b63e-429b-9968-b77b71cff497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:37.645083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c8d66-b63e-429b-9968-b77b71cff497 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:37.650006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:37.650715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:37.651090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:37.651501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] PATH_INFO: `/v3/projects/86e39994e8db4f128d5a6183b8d91cd6` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:37.662848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86e39994e8db4f128d5a6183b8d91cd6)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:37.662848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:37.846514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86e39994e8db4f128d5a6183b8d91cd6', 'request_id': 'req-55ac8fb8-b8af-44dd-93f2-f520c7534656'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:37.853306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86e39994e8db4f128d5a6183b8d91cd6', 'request_id': 'req-55ac8fb8-b8af-44dd-93f2-f520c7534656'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:37.859066 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86e39994e8db4f128d5a6183b8d91cd6', 'request_id': 'req-55ac8fb8-b8af-44dd-93f2-f520c7534656'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:37.956016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] The token cache is being invalidate because project 86e39994e8db4f128d5a6183b8d91cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:37.956590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:37.957063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:37.958671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 916/1824] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:37 2022] DELETE /identity/v3/projects/86e39994e8db4f128d5a6183b8d91cd6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:37.962739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ed7f3-c723-4c7d-ba22-93e73559a5f0 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:37.973134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ed7f3-c723-4c7d-ba22-93e73559a5f0 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:38.037253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:38.037892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:38.038159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:38.038430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] PATH_INFO: `/v3/users/c55d94c8b87e4eb69a8ab330d3f9e1f3` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:38.039661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c55d94c8b87e4eb69a8ab330d3f9e1f3)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:38.041622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:38.066518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:38.327685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c55d94c8b87e4eb69a8ab330d3f9e1f3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:38.328342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c55d94c8b87e4eb69a8ab330d3f9e1f3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:38.329068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c55d94c8b87e4eb69a8ab330d3f9e1f3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:38.445767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c55d94c8b87e4eb69a8ab330d3f9e1f3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:38.459315 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c55d94c8b87e4eb69a8ab330d3f9e1f3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:38.471953 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 909/1825] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:37 2022] DELETE /identity/v3/users/c55d94c8b87e4eb69a8ab330d3f9e1f3 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:38.967100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:38.968570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:38.990770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac8fb8-b8af-44dd-93f2-f520c7534656 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:39.053323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:39.053968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:39.054251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:39.054527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] PATH_INFO: `/v3/projects/e962c399b4c14a72b5e51c6a9cb1e148` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:39.055843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e962c399b4c14a72b5e51c6a9cb1e148)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:39.057766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:39.197201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e962c399b4c14a72b5e51c6a9cb1e148', 'request_id': 'req-9f07bc0b-fe77-42d3-94d5-1e67101052c1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:39.203335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e962c399b4c14a72b5e51c6a9cb1e148', 'request_id': 'req-9f07bc0b-fe77-42d3-94d5-1e67101052c1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:39.209633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e962c399b4c14a72b5e51c6a9cb1e148', 'request_id': 'req-9f07bc0b-fe77-42d3-94d5-1e67101052c1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:39.211578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:39.327364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] The token cache is being invalidate because project e962c399b4c14a72b5e51c6a9cb1e148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:39.327742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:39.328482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:39.331090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 917/1826] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:38 2022] DELETE /identity/v3/projects/e962c399b4c14a72b5e51c6a9cb1e148 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:39.335267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:39.337678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:39.353088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bdf397-e2f4-446c-9b12-a6b610678d01 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:39.421790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:39.422614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:39.422891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:39.423217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] PATH_INFO: `/v3/users/352865e49b7b41df8a8fa0bf9d2c5359` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:39.424819 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=352865e49b7b41df8a8fa0bf9d2c5359)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:39.426796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:39.670823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '352865e49b7b41df8a8fa0bf9d2c5359'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:39.671511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '352865e49b7b41df8a8fa0bf9d2c5359'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:39.672418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '352865e49b7b41df8a8fa0bf9d2c5359'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:39.765107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '352865e49b7b41df8a8fa0bf9d2c5359'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:39.777818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8c6b32fc-61c0-43a6-9d55-3bc590195738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '352865e49b7b41df8a8fa0bf9d2c5359'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:39.790002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 910/1827] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:39 2022] DELETE /identity/v3/users/352865e49b7b41df8a8fa0bf9d2c5359 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:40.346533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:40.348035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:40.360904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f07bc0b-fe77-42d3-94d5-1e67101052c1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:40.471865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:40.472668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:40.478309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:40.478309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] PATH_INFO: `/v3/projects/c664152846de4817a0c767c1de7960e1` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:40.478309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c664152846de4817a0c767c1de7960e1)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:40.480249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:40.660223 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c664152846de4817a0c767c1de7960e1', 'request_id': 'req-9a0759f6-f448-49d9-8811-feb3ced0e8e9'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:40.674736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c664152846de4817a0c767c1de7960e1', 'request_id': 'req-9a0759f6-f448-49d9-8811-feb3ced0e8e9'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:40.687921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c664152846de4817a0c767c1de7960e1', 'request_id': 'req-9a0759f6-f448-49d9-8811-feb3ced0e8e9'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:40.777080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] The token cache is being invalidate because project c664152846de4817a0c767c1de7960e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:40.777431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:40.778051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a0759f6-f448-49d9-8811-feb3ced0e8e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:40.780428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 918/1828] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:40 2022] DELETE /identity/v3/projects/c664152846de4817a0c767c1de7960e1 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:40.787495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af464b31-2fc5-4d5c-ba55-615c348780a0 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:40.790267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af464b31-2fc5-4d5c-ba55-615c348780a0 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:40.790267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af464b31-2fc5-4d5c-ba55-615c348780a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:40.816805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-af464b31-2fc5-4d5c-ba55-615c348780a0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:40.818237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-af464b31-2fc5-4d5c-ba55-615c348780a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:40.838737 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:40.887982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 911/1829] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:40.897075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257ce56-a581-4efd-b909-d010e0f8d0f2 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:40.897622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257ce56-a581-4efd-b909-d010e0f8d0f2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:40.897901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257ce56-a581-4efd-b909-d010e0f8d0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:40.919843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-e257ce56-a581-4efd-b909-d010e0f8d0f2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:40.921054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e257ce56-a581-4efd-b909-d010e0f8d0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:40.930238 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 919/1830] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:40.934491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af464b31-2fc5-4d5c-ba55-615c348780a0 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:40.944713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af464b31-2fc5-4d5c-ba55-615c348780a0 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:40.951719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7df2a-2c27-4fcf-8c27-b8b0f8906c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:40.952323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7df2a-2c27-4fcf-8c27-b8b0f8906c72 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:40.952596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7df2a-2c27-4fcf-8c27-b8b0f8906c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:40.952876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7df2a-2c27-4fcf-8c27-b8b0f8906c72 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:40.953499 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7df2a-2c27-4fcf-8c27-b8b0f8906c72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:00:40.953921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7df2a-2c27-4fcf-8c27-b8b0f8906c72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:40.955950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7df2a-2c27-4fcf-8c27-b8b0f8906c72 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:40.971538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 912/1831] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:00:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:00:40.974776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257ce56-a581-4efd-b909-d010e0f8d0f2 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:40.984364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257ce56-a581-4efd-b909-d010e0f8d0f2 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:40.989381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edad312-da5e-4c75-a8fd-4352e3c418a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:40.991062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edad312-da5e-4c75-a8fd-4352e3c418a0 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:40.991062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edad312-da5e-4c75-a8fd-4352e3c418a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:40.991062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edad312-da5e-4c75-a8fd-4352e3c418a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:40.991315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edad312-da5e-4c75-a8fd-4352e3c418a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:40.993261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edad312-da5e-4c75-a8fd-4352e3c418a0 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:41.150216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 920/1832] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:40 2022] POST /identity/v3/projects => generated 376 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:41.154137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7df2a-2c27-4fcf-8c27-b8b0f8906c72 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:41.164216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7df2a-2c27-4fcf-8c27-b8b0f8906c72 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:41.220665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1c67a-f175-4017-bb54-b0f0017b493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:41.221262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1c67a-f175-4017-bb54-b0f0017b493f admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:41.221537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1c67a-f175-4017-bb54-b0f0017b493f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:41.221802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1c67a-f175-4017-bb54-b0f0017b493f admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:41.222417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1c67a-f175-4017-bb54-b0f0017b493f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:41.224371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1c67a-f175-4017-bb54-b0f0017b493f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:41.383841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 913/1833] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:41 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:41.387739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edad312-da5e-4c75-a8fd-4352e3c418a0 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:41.398743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edad312-da5e-4c75-a8fd-4352e3c418a0 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:41.403614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55907b4-8d56-420e-bc7b-7d01225f1d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:41.404150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55907b4-8d56-420e-bc7b-7d01225f1d12 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:41.404426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55907b4-8d56-420e-bc7b-7d01225f1d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:41.404696 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55907b4-8d56-420e-bc7b-7d01225f1d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:41.405335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55907b4-8d56-420e-bc7b-7d01225f1d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:41.406915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55907b4-8d56-420e-bc7b-7d01225f1d12 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:41.417394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 921/1834] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:41 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:41.421678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1c67a-f175-4017-bb54-b0f0017b493f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:41.433658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1c67a-f175-4017-bb54-b0f0017b493f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:41.438320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e399ac7-9ec9-4a48-bc2f-6a35deac8726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:41.439837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399ac7-9ec9-4a48-bc2f-6a35deac8726 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:41.440091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399ac7-9ec9-4a48-bc2f-6a35deac8726 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:41.440619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e399ac7-9ec9-4a48-bc2f-6a35deac8726 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:41.441418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e399ac7-9ec9-4a48-bc2f-6a35deac8726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:41.443163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e399ac7-9ec9-4a48-bc2f-6a35deac8726 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:41.456425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 914/1835] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:41 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:00:41.462295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55907b4-8d56-420e-bc7b-7d01225f1d12 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:41.477096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55907b4-8d56-420e-bc7b-7d01225f1d12 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:41.484650 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf50430-3165-49d1-b8dc-567f4a7e63d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:41.486266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf50430-3165-49d1-b8dc-567f4a7e63d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:41.487599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf50430-3165-49d1-b8dc-567f4a7e63d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:41.487950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf50430-3165-49d1-b8dc-567f4a7e63d2 admin admin] PATH_INFO: `/v3/projects/db4a121febfd42169e12a97e97082b59/users/83dc4238b541416689ec46bdf8c96728/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:41.510120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf50430-3165-49d1-b8dc-567f4a7e63d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db4a121febfd42169e12a97e97082b59, user_id=83dc4238b541416689ec46bdf8c96728, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:41.516665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf50430-3165-49d1-b8dc-567f4a7e63d2 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:41.640058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 922/1836] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:41 2022] PUT /identity/v3/projects/db4a121febfd42169e12a97e97082b59/users/83dc4238b541416689ec46bdf8c96728/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:41.651806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82ce09-9460-4cd8-bbbd-1c153e1193b5 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:41.652271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82ce09-9460-4cd8-bbbd-1c153e1193b5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:41.652693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82ce09-9460-4cd8-bbbd-1c153e1193b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:41.685748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8c82ce09-9460-4cd8-bbbd-1c153e1193b5 None None] MFA Rules not processed for user `83dc4238b541416689ec46bdf8c96728`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:41.686092 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8c82ce09-9460-4cd8-bbbd-1c153e1193b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:41.748861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 915/1837] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:41.753849 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf50430-3165-49d1-b8dc-567f4a7e63d2 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:41.773074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:41.816433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf50430-3165-49d1-b8dc-567f4a7e63d2 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:41.821061 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211fa39-c3cd-4da6-aef5-e25d8bee14a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:41.821657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211fa39-c3cd-4da6-aef5-e25d8bee14a4 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:41.821932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211fa39-c3cd-4da6-aef5-e25d8bee14a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:41.822198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211fa39-c3cd-4da6-aef5-e25d8bee14a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:41.822828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211fa39-c3cd-4da6-aef5-e25d8bee14a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:41.824732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211fa39-c3cd-4da6-aef5-e25d8bee14a4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:41.939562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 923/1838] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:41 2022] POST /identity/v3/projects => generated 374 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:41.943374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82ce09-9460-4cd8-bbbd-1c153e1193b5 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:41.954920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82ce09-9460-4cd8-bbbd-1c153e1193b5 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:41.977856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:42.054470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0fc90-1b4d-41b0-9eae-5c7446ac750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:42.055709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0fc90-1b4d-41b0-9eae-5c7446ac750d admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:42.055989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0fc90-1b4d-41b0-9eae-5c7446ac750d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:42.056423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0fc90-1b4d-41b0-9eae-5c7446ac750d admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:42.057232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0fc90-1b4d-41b0-9eae-5c7446ac750d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:42.059253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0fc90-1b4d-41b0-9eae-5c7446ac750d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:42.201232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211fa39-c3cd-4da6-aef5-e25d8bee14a4 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:42.207691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 916/1839] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:41 2022] POST /identity/v3/users => generated 294 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:42.212844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211fa39-c3cd-4da6-aef5-e25d8bee14a4 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:42.218976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f78a5-28dd-4453-b870-562d0971e850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:42.221528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f78a5-28dd-4453-b870-562d0971e850 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:42.221528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f78a5-28dd-4453-b870-562d0971e850 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:42.221528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f78a5-28dd-4453-b870-562d0971e850 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:42.221943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f78a5-28dd-4453-b870-562d0971e850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:42.223743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f78a5-28dd-4453-b870-562d0971e850 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:42.235213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 924/1840] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:42 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:42.240011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0fc90-1b4d-41b0-9eae-5c7446ac750d admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:42.251281 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0fc90-1b4d-41b0-9eae-5c7446ac750d admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:42.257182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfaf62d-d64f-4a4a-a360-059340745cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:42.257953 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfaf62d-d64f-4a4a-a360-059340745cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:42.258294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfaf62d-d64f-4a4a-a360-059340745cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:42.258626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfaf62d-d64f-4a4a-a360-059340745cc5 admin admin] PATH_INFO: `/v3/projects/6c52be5c5b8440258e12880ef848b128/users/6bcbd767fc434c3099ab01c02e0abe9b/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:42.280705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfaf62d-d64f-4a4a-a360-059340745cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c52be5c5b8440258e12880ef848b128, user_id=6bcbd767fc434c3099ab01c02e0abe9b, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:42.284758 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfaf62d-d64f-4a4a-a360-059340745cc5 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:42.377768 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 917/1841] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:42 2022] PUT /identity/v3/projects/6c52be5c5b8440258e12880ef848b128/users/6bcbd767fc434c3099ab01c02e0abe9b/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:42.381715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f78a5-28dd-4453-b870-562d0971e850 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:42.391491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f78a5-28dd-4453-b870-562d0971e850 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:42.448476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb845347-be7e-4437-938c-e4a340d4942f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:42.449118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb845347-be7e-4437-938c-e4a340d4942f admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:42.449361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb845347-be7e-4437-938c-e4a340d4942f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:42.449638 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb845347-be7e-4437-938c-e4a340d4942f admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:42.450311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb845347-be7e-4437-938c-e4a340d4942f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:42.451973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb845347-be7e-4437-938c-e4a340d4942f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:42.466008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 925/1842] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:42 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:42.467259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfaf62d-d64f-4a4a-a360-059340745cc5 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:42.477586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfaf62d-d64f-4a4a-a360-059340745cc5 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:42.483066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf9a85-e95f-4f4a-aa3b-d4183fdb8f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:42.483778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf9a85-e95f-4f4a-aa3b-d4183fdb8f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:42.484057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf9a85-e95f-4f4a-aa3b-d4183fdb8f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:42.484329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebf9a85-e95f-4f4a-aa3b-d4183fdb8f91 admin admin] PATH_INFO: `/v3/projects/6c52be5c5b8440258e12880ef848b128/users/6bcbd767fc434c3099ab01c02e0abe9b/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:42.486573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf9a85-e95f-4f4a-aa3b-d4183fdb8f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c52be5c5b8440258e12880ef848b128, user_id=6bcbd767fc434c3099ab01c02e0abe9b, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:42.489294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebf9a85-e95f-4f4a-aa3b-d4183fdb8f91 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:42.500868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 918/1843] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:42 2022] PUT /identity/v3/projects/6c52be5c5b8440258e12880ef848b128/users/6bcbd767fc434c3099ab01c02e0abe9b/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:42.512549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6342c2-e6e2-4a26-83ef-4e28bcfb0dca None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:42.512838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6342c2-e6e2-4a26-83ef-4e28bcfb0dca None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:42.513113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6342c2-e6e2-4a26-83ef-4e28bcfb0dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:42.541100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-af6342c2-e6e2-4a26-83ef-4e28bcfb0dca None None] MFA Rules not processed for user `6bcbd767fc434c3099ab01c02e0abe9b`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:42.542341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-af6342c2-e6e2-4a26-83ef-4e28bcfb0dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:42.612920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 926/1844] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:42.624392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf9a85-e95f-4f4a-aa3b-d4183fdb8f91 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:42.626096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0ebf9a85-e95f-4f4a-aa3b-d4183fdb8f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:42.683872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebf9a85-e95f-4f4a-aa3b-d4183fdb8f91 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:42.687638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945bb84a-bbcc-47c4-b0d9-575066880dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:42.688192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bb84a-bbcc-47c4-b0d9-575066880dbe service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:42.688473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bb84a-bbcc-47c4-b0d9-575066880dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:42.689285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945bb84a-bbcc-47c4-b0d9-575066880dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:42.754081 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bb84a-bbcc-47c4-b0d9-575066880dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:42.755843 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945bb84a-bbcc-47c4-b0d9-575066880dbe service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:42.765728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 919/1845] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:49.649587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6342c2-e6e2-4a26-83ef-4e28bcfb0dca None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:49.666766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:49.744195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6342c2-e6e2-4a26-83ef-4e28bcfb0dca None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:49.749479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:49.750424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:49.750738 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:49.751011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] PATH_INFO: `/v3/users/83dc4238b541416689ec46bdf8c96728` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:49.752519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83dc4238b541416689ec46bdf8c96728)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:49.754338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:50.091835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83dc4238b541416689ec46bdf8c96728'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:50.092576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83dc4238b541416689ec46bdf8c96728'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:50.093263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83dc4238b541416689ec46bdf8c96728'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:50.227952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83dc4238b541416689ec46bdf8c96728'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:50.245831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83dc4238b541416689ec46bdf8c96728'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:50.263513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 927/1846] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:49 2022] DELETE /identity/v3/users/83dc4238b541416689ec46bdf8c96728 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:50.269416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1afa8-3861-4d47-aaf1-5955393b863f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:50.269905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1afa8-3861-4d47-aaf1-5955393b863f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:50.270163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1afa8-3861-4d47-aaf1-5955393b863f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:50.293589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5ae1afa8-3861-4d47-aaf1-5955393b863f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:50.294784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5ae1afa8-3861-4d47-aaf1-5955393b863f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:50.314055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:50.356284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 920/1847] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:50.367155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:50.368624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:50.433421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c229a2-df5f-497d-adcc-c966a27ad316 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:50.437204 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d5d3-f8a0-4cfc-a9bb-b8456c1cf6f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:50.437827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d5d3-f8a0-4cfc-a9bb-b8456c1cf6f5 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:50.438769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d5d3-f8a0-4cfc-a9bb-b8456c1cf6f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:50.439084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d5d3-f8a0-4cfc-a9bb-b8456c1cf6f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:50.449678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d5d3-f8a0-4cfc-a9bb-b8456c1cf6f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:50.451350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d5d3-f8a0-4cfc-a9bb-b8456c1cf6f5 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:50.463206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 928/1848] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:50.531821 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1afa8-3861-4d47-aaf1-5955393b863f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:50.533467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5ae1afa8-3861-4d47-aaf1-5955393b863f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:50.551995 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1afa8-3861-4d47-aaf1-5955393b863f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:50.558052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:50.559329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:50.560092 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:50.560554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] PATH_INFO: `/v3/users/2d9f1ce89ead453abf5a0b431e07f888` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:50.565201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d9f1ce89ead453abf5a0b431e07f888)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:50.569156 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:50.870402 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d9f1ce89ead453abf5a0b431e07f888'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:50.871419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d9f1ce89ead453abf5a0b431e07f888'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:50.872172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d9f1ce89ead453abf5a0b431e07f888'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:51.017711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d9f1ce89ead453abf5a0b431e07f888'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:51.034610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d9f1ce89ead453abf5a0b431e07f888'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:51.085735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 921/1849] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:50 2022] DELETE /identity/v3/users/2d9f1ce89ead453abf5a0b431e07f888 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:51.085978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b8fa9-1794-4cb9-a99a-dfa81dc4ccb9 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:51.085978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b8fa9-1794-4cb9-a99a-dfa81dc4ccb9 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:51.085978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b8fa9-1794-4cb9-a99a-dfa81dc4ccb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:51.115271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-047b8fa9-1794-4cb9-a99a-dfa81dc4ccb9 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:51.116608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-047b8fa9-1794-4cb9-a99a-dfa81dc4ccb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:51.148626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:51.180640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 929/1850] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:51.188690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:51.190323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:51.254764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9bda8a-461b-4347-a36e-f73252b1f607 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:51.258323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5431c9b-3b64-4563-a6a5-4a2853d4fafb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:51.258925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431c9b-3b64-4563-a6a5-4a2853d4fafb service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:51.259234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431c9b-3b64-4563-a6a5-4a2853d4fafb service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:51.259562 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5431c9b-3b64-4563-a6a5-4a2853d4fafb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:51.271710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5431c9b-3b64-4563-a6a5-4a2853d4fafb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:51.275048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5431c9b-3b64-4563-a6a5-4a2853d4fafb service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:51.276528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b8fa9-1794-4cb9-a99a-dfa81dc4ccb9 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:51.278001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-047b8fa9-1794-4cb9-a99a-dfa81dc4ccb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:51.286547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 922/1851] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:00:51.292762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b8fa9-1794-4cb9-a99a-dfa81dc4ccb9 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:51.297827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:51.298538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:51.298884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:51.299290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] PATH_INFO: `/v3/projects/db4a121febfd42169e12a97e97082b59` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:51.300539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db4a121febfd42169e12a97e97082b59)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:51.302788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:51.499007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db4a121febfd42169e12a97e97082b59', 'request_id': 'req-d8ac4d4b-26a8-4b38-819e-971576726130'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:51.506820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db4a121febfd42169e12a97e97082b59', 'request_id': 'req-d8ac4d4b-26a8-4b38-819e-971576726130'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:51.516129 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db4a121febfd42169e12a97e97082b59', 'request_id': 'req-d8ac4d4b-26a8-4b38-819e-971576726130'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:51.724075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] The token cache is being invalidate because project db4a121febfd42169e12a97e97082b59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:51.724404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:51.725028 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:51.731676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 930/1852] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:51 2022] DELETE /identity/v3/projects/db4a121febfd42169e12a97e97082b59 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:51.734338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5431c9b-3b64-4563-a6a5-4a2853d4fafb service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:51.747132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5431c9b-3b64-4563-a6a5-4a2853d4fafb service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:51.787755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:51.814933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:51.817275 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:51.817469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:51.817756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] PATH_INFO: `/v3/users/6bcbd767fc434c3099ab01c02e0abe9b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:51.818984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bcbd767fc434c3099ab01c02e0abe9b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:51.820999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:52.198781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:52.200956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:52.217326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac4d4b-26a8-4b38-819e-971576726130 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:52.225731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:52.226818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:52.227286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:52.231358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] PATH_INFO: `/v3/projects/a26395fa888743908b6b32261d80dc43` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:52.231358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26395fa888743908b6b32261d80dc43)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:52.240075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:52.281449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bcbd767fc434c3099ab01c02e0abe9b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.283548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bcbd767fc434c3099ab01c02e0abe9b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.284381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bcbd767fc434c3099ab01c02e0abe9b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.384615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26395fa888743908b6b32261d80dc43', 'request_id': 'req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.391164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26395fa888743908b6b32261d80dc43', 'request_id': 'req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.396823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26395fa888743908b6b32261d80dc43', 'request_id': 'req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.399396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bcbd767fc434c3099ab01c02e0abe9b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.417516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bcbd767fc434c3099ab01c02e0abe9b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.438727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 923/1853] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:51 2022] DELETE /identity/v3/users/6bcbd767fc434c3099ab01c02e0abe9b => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:52.623888 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] The token cache is being invalidate because project a26395fa888743908b6b32261d80dc43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:52.626770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.626770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.627683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 931/1854] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:52 2022] DELETE /identity/v3/projects/a26395fa888743908b6b32261d80dc43 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:52.631849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:52.633537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:52.700908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194225-33b7-42b1-9230-34453274d968 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:52.706245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:52.718626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:52.718626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:52.718626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] PATH_INFO: `/v3/users/a6a942b6d0184ca190304fe847224a0a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:52.718626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a942b6d0184ca190304fe847224a0a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:52.718626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:52.995792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a942b6d0184ca190304fe847224a0a'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.996450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a942b6d0184ca190304fe847224a0a'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:52.997053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a942b6d0184ca190304fe847224a0a'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:53.069581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:53.071229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:53.090822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a942b6d0184ca190304fe847224a0a'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:53.114535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a942b6d0184ca190304fe847224a0a'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:53.130862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 924/1855] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:52 2022] DELETE /identity/v3/users/a6a942b6d0184ca190304fe847224a0a => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:53.144247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:53.164932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8178f4-d80a-4a81-bc54-dfe22dcc92e0 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:53.241274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:53.241957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:53.242189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:53.242461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] PATH_INFO: `/v3/projects/6c52be5c5b8440258e12880ef848b128` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:53.244552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c52be5c5b8440258e12880ef848b128)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:53.246746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:53.490010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c52be5c5b8440258e12880ef848b128', 'request_id': 'req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:53.499584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c52be5c5b8440258e12880ef848b128', 'request_id': 'req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:53.508527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c52be5c5b8440258e12880ef848b128', 'request_id': 'req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:53.689122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:53.694717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:53.703241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] The token cache is being invalidate because project 6c52be5c5b8440258e12880ef848b128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:53.703241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:53.703241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5850a84f-6f7e-44dd-b9b0-b2ef263a7224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:53.703241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 932/1856] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:53 2022] DELETE /identity/v3/projects/6c52be5c5b8440258e12880ef848b128 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:53.708581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aca9d-8eba-4069-af35-3d7741df31d0 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:53.710744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aca9d-8eba-4069-af35-3d7741df31d0 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:53.711244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3aca9d-8eba-4069-af35-3d7741df31d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:53.711783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2face468-11c7-4ea3-b400-d75cff64a903 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:53.742975 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5f3aca9d-8eba-4069-af35-3d7741df31d0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:53.748877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5f3aca9d-8eba-4069-af35-3d7741df31d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:53.778388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:53.779076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:53.779555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:53.779833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] PATH_INFO: `/v3/users/33e7a831c55e41e3bf6aa0921c92ce87` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:53.781311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33e7a831c55e41e3bf6aa0921c92ce87)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:53.783344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:53.821246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 933/1857] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:00:53.830147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a29259-4e61-45aa-bc2b-45c4a649daf3 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:53.830433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a29259-4e61-45aa-bc2b-45c4a649daf3 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:53.830707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a29259-4e61-45aa-bc2b-45c4a649daf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:53.852606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b5a29259-4e61-45aa-bc2b-45c4a649daf3 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:53.854157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b5a29259-4e61-45aa-bc2b-45c4a649daf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:53.862450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 934/1858] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:00:53.868110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a29259-4e61-45aa-bc2b-45c4a649daf3 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:53.879751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a29259-4e61-45aa-bc2b-45c4a649daf3 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:53.886363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6a267-dd85-4c67-9177-b20cdf7fab0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:53.886978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a267-dd85-4c67-9177-b20cdf7fab0a admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:53.887286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a267-dd85-4c67-9177-b20cdf7fab0a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:53.887592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6a267-dd85-4c67-9177-b20cdf7fab0a admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:53.888191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6a267-dd85-4c67-9177-b20cdf7fab0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:00:53.888490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6a267-dd85-4c67-9177-b20cdf7fab0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:53.890082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6a267-dd85-4c67-9177-b20cdf7fab0a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:53.905548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 935/1859] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:00:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:00:53.910519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6a267-dd85-4c67-9177-b20cdf7fab0a admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:53.912464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ebe6a267-dd85-4c67-9177-b20cdf7fab0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:53.927250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6a267-dd85-4c67-9177-b20cdf7fab0a admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:53.932085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:53.932842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:53.933180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:53.933402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] PATH_INFO: `/v3/projects/dcc679824bc34f4dafbd03891462278a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:53.934538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc679824bc34f4dafbd03891462278a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:53.936481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:54.054750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33e7a831c55e41e3bf6aa0921c92ce87'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:54.055523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33e7a831c55e41e3bf6aa0921c92ce87'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:54.056130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33e7a831c55e41e3bf6aa0921c92ce87'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:54.057735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc679824bc34f4dafbd03891462278a', 'request_id': 'req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:54.067221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc679824bc34f4dafbd03891462278a', 'request_id': 'req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:54.072701 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc679824bc34f4dafbd03891462278a', 'request_id': 'req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:54.113690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:54.192902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33e7a831c55e41e3bf6aa0921c92ce87'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:54.207784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33e7a831c55e41e3bf6aa0921c92ce87'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:54.220790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 925/1860] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:00:53 2022] DELETE /identity/v3/users/33e7a831c55e41e3bf6aa0921c92ce87 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:54.224346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:54.225707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:54.301785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] The token cache is being invalidate because project dcc679824bc34f4dafbd03891462278a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:54.302629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:54.303296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d67e454-d4d7-4b4c-bbe1-2259ffad1b24 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:54.303425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-ce35a7d8-b7c4-4df6-9a70-8328abb610f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:54.304949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 936/1861] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:53 2022] DELETE /identity/v3/projects/dcc679824bc34f4dafbd03891462278a => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:54.306473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b707778-efb5-4723-a582-7410cc11b32c None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:54.307123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b707778-efb5-4723-a582-7410cc11b32c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:54.307694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b707778-efb5-4723-a582-7410cc11b32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:54.312563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa878371-03d4-4f6f-81c1-caf35b12b827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:54.313313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa878371-03d4-4f6f-81c1-caf35b12b827 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:54.313598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa878371-03d4-4f6f-81c1-caf35b12b827 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:54.314307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa878371-03d4-4f6f-81c1-caf35b12b827 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:54.314526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa878371-03d4-4f6f-81c1-caf35b12b827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:54.317050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa878371-03d4-4f6f-81c1-caf35b12b827 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:54.332087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2b707778-efb5-4723-a582-7410cc11b32c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:54.333849 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2b707778-efb5-4723-a582-7410cc11b32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:54.342718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 937/1862] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:54.344706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de7117-6ff1-44bc-8526-57e7493130d2 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:54.345195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de7117-6ff1-44bc-8526-57e7493130d2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:54.345599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de7117-6ff1-44bc-8526-57e7493130d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:54.370020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-30de7117-6ff1-44bc-8526-57e7493130d2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:54.371189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-30de7117-6ff1-44bc-8526-57e7493130d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:54.381146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 938/1863] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:54.381999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de7117-6ff1-44bc-8526-57e7493130d2 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:54.383330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-30de7117-6ff1-44bc-8526-57e7493130d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:54.443156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de7117-6ff1-44bc-8526-57e7493130d2 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:54.446711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca98278-6649-4017-898a-74617db3c6a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:54.447352 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca98278-6649-4017-898a-74617db3c6a6 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:54.447631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca98278-6649-4017-898a-74617db3c6a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:54.447898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca98278-6649-4017-898a-74617db3c6a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:54.460761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca98278-6649-4017-898a-74617db3c6a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:54.462195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca98278-6649-4017-898a-74617db3c6a6 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:54.466171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 926/1864] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:54 2022] POST /identity/v3/projects => generated 372 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:54.466171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f4ef-06e1-4c8c-b955-3f96cfc768d6 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:54.466171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f4ef-06e1-4c8c-b955-3f96cfc768d6 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:54.466171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460f4ef-06e1-4c8c-b955-3f96cfc768d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:54.473625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 939/1865] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:54.474627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca98278-6649-4017-898a-74617db3c6a6 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:54.476581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fca98278-6649-4017-898a-74617db3c6a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:54.496176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca98278-6649-4017-898a-74617db3c6a6 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:54.513239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2460f4ef-06e1-4c8c-b955-3f96cfc768d6 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:54.514685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2460f4ef-06e1-4c8c-b955-3f96cfc768d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:54.586048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed6795-422d-42a5-9ff2-572e3d75cff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:54.587466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed6795-422d-42a5-9ff2-572e3d75cff4 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:54.587977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed6795-422d-42a5-9ff2-572e3d75cff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:54.588465 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed6795-422d-42a5-9ff2-572e3d75cff4 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:54.589476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed6795-422d-42a5-9ff2-572e3d75cff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:54.592064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ed6795-422d-42a5-9ff2-572e3d75cff4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:54.613275 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 927/1866] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:54.620301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460f4ef-06e1-4c8c-b955-3f96cfc768d6 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:54.632645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460f4ef-06e1-4c8c-b955-3f96cfc768d6 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:54.638738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719ef64-dd3d-49c8-a2d7-49670d303a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:54.639476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719ef64-dd3d-49c8-a2d7-49670d303a93 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:54.639840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719ef64-dd3d-49c8-a2d7-49670d303a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:54.640196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4719ef64-dd3d-49c8-a2d7-49670d303a93 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:54.641477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719ef64-dd3d-49c8-a2d7-49670d303a93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:00:54.641932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719ef64-dd3d-49c8-a2d7-49670d303a93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:54.643691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4719ef64-dd3d-49c8-a2d7-49670d303a93 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:54.662828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 928/1867] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:00:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:00:54.664058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719ef64-dd3d-49c8-a2d7-49670d303a93 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:54.675616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4719ef64-dd3d-49c8-a2d7-49670d303a93 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:54.680942 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bec22-3b57-40e8-a980-388489429198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:54.681713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bec22-3b57-40e8-a980-388489429198 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:54.682087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bec22-3b57-40e8-a980-388489429198 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:54.682423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bec22-3b57-40e8-a980-388489429198 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:54.683175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bec22-3b57-40e8-a980-388489429198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:54.685154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bec22-3b57-40e8-a980-388489429198 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:54.821803 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 940/1868] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:54 2022] POST /identity/v3/users => generated 287 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:54.823130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed6795-422d-42a5-9ff2-572e3d75cff4 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:54.835711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed6795-422d-42a5-9ff2-572e3d75cff4 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:54.841285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d006b-8cb7-4041-a035-e9f9945e3ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:54.842592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d006b-8cb7-4041-a035-e9f9945e3ac6 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:54.842592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d006b-8cb7-4041-a035-e9f9945e3ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:54.842793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468d006b-8cb7-4041-a035-e9f9945e3ac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:54.843480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d006b-8cb7-4041-a035-e9f9945e3ac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:54.845516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468d006b-8cb7-4041-a035-e9f9945e3ac6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:54.872600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 941/1869] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:54 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:54.880734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 929/1870] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:54 2022] POST /identity/v3/projects => generated 366 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:54.881261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d006b-8cb7-4041-a035-e9f9945e3ac6 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:54.883001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bec22-3b57-40e8-a980-388489429198 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:54.895251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468d006b-8cb7-4041-a035-e9f9945e3ac6 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:54.986833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85cc79-f2f1-4db6-a58a-f3916a3ecfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:54.986833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85cc79-f2f1-4db6-a58a-f3916a3ecfa1 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:54.986833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85cc79-f2f1-4db6-a58a-f3916a3ecfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:54.986833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85cc79-f2f1-4db6-a58a-f3916a3ecfa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:54.987398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85cc79-f2f1-4db6-a58a-f3916a3ecfa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:54.989641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85cc79-f2f1-4db6-a58a-f3916a3ecfa1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:55.002965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bec22-3b57-40e8-a980-388489429198 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:55.015491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c279e-16f1-401c-8dfe-d96312f9cc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:55.025307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c279e-16f1-401c-8dfe-d96312f9cc3a admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:55.025307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c279e-16f1-401c-8dfe-d96312f9cc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:55.025307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c279e-16f1-401c-8dfe-d96312f9cc3a admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:55.025307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c279e-16f1-401c-8dfe-d96312f9cc3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:55.025307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c279e-16f1-401c-8dfe-d96312f9cc3a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:55.025894 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 942/1871] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:54 2022] GET /identity/v3/roles => generated 1427 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:55.029125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85cc79-f2f1-4db6-a58a-f3916a3ecfa1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:55.042496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85cc79-f2f1-4db6-a58a-f3916a3ecfa1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:55.049713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9e2da-f8fa-4c3f-9c4f-35d21082fd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:55.050048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9e2da-f8fa-4c3f-9c4f-35d21082fd54 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:55.051054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9e2da-f8fa-4c3f-9c4f-35d21082fd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:55.051054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9e2da-f8fa-4c3f-9c4f-35d21082fd54 admin admin] PATH_INFO: `/v3/projects/6458afad3a654baebc09ce00a9d695bb/users/946a7533230b446f8ff34a0a51d4c2f2/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:55.072659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9e2da-f8fa-4c3f-9c4f-35d21082fd54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6458afad3a654baebc09ce00a9d695bb, user_id=946a7533230b446f8ff34a0a51d4c2f2, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:55.075710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9e2da-f8fa-4c3f-9c4f-35d21082fd54 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:55.207472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 943/1872] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:55 2022] PUT /identity/v3/projects/6458afad3a654baebc09ce00a9d695bb/users/946a7533230b446f8ff34a0a51d4c2f2/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:55.209048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1cc0d-460f-406b-8c13-c70eaab2437d None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:55.209342 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1cc0d-460f-406b-8c13-c70eaab2437d None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:55.209715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1cc0d-460f-406b-8c13-c70eaab2437d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:55.212502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 930/1873] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:54 2022] POST /identity/v3/users => generated 284 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:55.213291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c279e-16f1-401c-8dfe-d96312f9cc3a admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:55.231624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c279e-16f1-401c-8dfe-d96312f9cc3a admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:55.237851 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5ed1cc0d-460f-406b-8c13-c70eaab2437d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:55.239180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5ed1cc0d-460f-406b-8c13-c70eaab2437d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:55.317021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c29db9-3ae9-4241-948b-43d72de44fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:55.317836 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c29db9-3ae9-4241-948b-43d72de44fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:55.318262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c29db9-3ae9-4241-948b-43d72de44fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:55.318694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c29db9-3ae9-4241-948b-43d72de44fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:55.319570 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c29db9-3ae9-4241-948b-43d72de44fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:55.321442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c29db9-3ae9-4241-948b-43d72de44fd2 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:55.336016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 931/1874] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:55 2022] GET /identity/v3/roles => generated 1427 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:55.340079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c29db9-3ae9-4241-948b-43d72de44fd2 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:55.345114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 944/1875] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:55.354870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c29db9-3ae9-4241-948b-43d72de44fd2 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:55.364004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1cc0d-460f-406b-8c13-c70eaab2437d None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:55.367010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b7e84-8026-4678-ab12-ebff037794ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:55.367706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b7e84-8026-4678-ab12-ebff037794ba admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:55.367987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b7e84-8026-4678-ab12-ebff037794ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:55.368354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b7e84-8026-4678-ab12-ebff037794ba admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:55.369001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b7e84-8026-4678-ab12-ebff037794ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:55.370844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b7e84-8026-4678-ab12-ebff037794ba admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:55.385195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 932/1876] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:55 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:55.390581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b7e84-8026-4678-ab12-ebff037794ba admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:55.404636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:55.407592 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b7e84-8026-4678-ab12-ebff037794ba admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:55.413348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11c4c8-38cb-47ac-b05a-5ae1c9fc467b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:55.414517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11c4c8-38cb-47ac-b05a-5ae1c9fc467b admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:55.414799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11c4c8-38cb-47ac-b05a-5ae1c9fc467b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:55.415117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11c4c8-38cb-47ac-b05a-5ae1c9fc467b admin admin] PATH_INFO: `/v3/projects/773d0469a77a47efb7563d3a83884d4d/users/c6c5c6109dda43a0a96ed15a9e535bbc/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:55.434010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11c4c8-38cb-47ac-b05a-5ae1c9fc467b admin admin] RBAC: Authorizing `identity:create_grant(project_id=773d0469a77a47efb7563d3a83884d4d, user_id=c6c5c6109dda43a0a96ed15a9e535bbc, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:55.437070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11c4c8-38cb-47ac-b05a-5ae1c9fc467b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:55.476686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1cc0d-460f-406b-8c13-c70eaab2437d None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:55.481619 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd06ef1-e004-4e94-ada8-a867df93020e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:55.482235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd06ef1-e004-4e94-ada8-a867df93020e service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:55.482561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd06ef1-e004-4e94-ada8-a867df93020e service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:55.482878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd06ef1-e004-4e94-ada8-a867df93020e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:55.495055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd06ef1-e004-4e94-ada8-a867df93020e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:55.496882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd06ef1-e004-4e94-ada8-a867df93020e service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:55.508203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 945/1877] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:55 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:55.564401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 933/1878] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:55 2022] PUT /identity/v3/projects/773d0469a77a47efb7563d3a83884d4d/users/c6c5c6109dda43a0a96ed15a9e535bbc/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:55.568454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69199a5a-50a5-4a5f-a2f2-29e6152b0e88 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:55.568741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69199a5a-50a5-4a5f-a2f2-29e6152b0e88 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:55.569069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69199a5a-50a5-4a5f-a2f2-29e6152b0e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:55.600919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-69199a5a-50a5-4a5f-a2f2-29e6152b0e88 None None] MFA Rules not processed for user `c6c5c6109dda43a0a96ed15a9e535bbc`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:55.602528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-69199a5a-50a5-4a5f-a2f2-29e6152b0e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:55.715966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 946/1879] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:55.720611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11c4c8-38cb-47ac-b05a-5ae1c9fc467b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:55.794933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11c4c8-38cb-47ac-b05a-5ae1c9fc467b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:55.800920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b756bc2-80f3-49c7-8494-0b24007fdb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:55.801908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b756bc2-80f3-49c7-8494-0b24007fdb6d admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:55.802284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b756bc2-80f3-49c7-8494-0b24007fdb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:55.802651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b756bc2-80f3-49c7-8494-0b24007fdb6d admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:55.803478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b756bc2-80f3-49c7-8494-0b24007fdb6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:55.805530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b756bc2-80f3-49c7-8494-0b24007fdb6d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:55.998850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 934/1880] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:55 2022] POST /identity/v3/projects => generated 366 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:56.000407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69199a5a-50a5-4a5f-a2f2-29e6152b0e88 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:56.010289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b756bc2-80f3-49c7-8494-0b24007fdb6d admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:56.012662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7b756bc2-80f3-49c7-8494-0b24007fdb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:56.013339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69199a5a-50a5-4a5f-a2f2-29e6152b0e88 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:56.139046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b756bc2-80f3-49c7-8494-0b24007fdb6d admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:56.144967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b137c5-caa0-4d44-be6b-2d0b52ddf254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:56.145596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b137c5-caa0-4d44-be6b-2d0b52ddf254 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.146540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b137c5-caa0-4d44-be6b-2d0b52ddf254 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.146822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b137c5-caa0-4d44-be6b-2d0b52ddf254 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.147692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b137c5-caa0-4d44-be6b-2d0b52ddf254 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:56.150734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b137c5-caa0-4d44-be6b-2d0b52ddf254 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:56.157076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:56.157752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.158182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.158355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] PATH_INFO: `/v3/projects/7b3d1c2e23a64b849d240f8a7ce9dcc0` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.159670 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3d1c2e23a64b849d240f8a7ce9dcc0)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:56.165560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:56.233630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:56.323600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 947/1881] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:55 2022] POST /identity/v3/users => generated 290 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:56.324816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b137c5-caa0-4d44-be6b-2d0b52ddf254 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:56.340622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b137c5-caa0-4d44-be6b-2d0b52ddf254 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:56.348275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d9bde-c03f-4e40-952b-d6e03c93d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:56.348932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d9bde-c03f-4e40-952b-d6e03c93d416 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.349170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d9bde-c03f-4e40-952b-d6e03c93d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.349452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d9bde-c03f-4e40-952b-d6e03c93d416 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.350159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d9bde-c03f-4e40-952b-d6e03c93d416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:56.361949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d9bde-c03f-4e40-952b-d6e03c93d416 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:56.379795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3d1c2e23a64b849d240f8a7ce9dcc0', 'request_id': 'req-63f62752-d9cd-4b29-9b52-98bd197057ef'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:56.388388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3d1c2e23a64b849d240f8a7ce9dcc0', 'request_id': 'req-63f62752-d9cd-4b29-9b52-98bd197057ef'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:56.393179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 948/1882] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:56 2022] GET /identity/v3/roles => generated 1427 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:56.397249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d9bde-c03f-4e40-952b-d6e03c93d416 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:56.421371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d9bde-c03f-4e40-952b-d6e03c93d416 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:56.505901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecae21c-45b7-4ffb-a8eb-f73058e627be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:56.506669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae21c-45b7-4ffb-a8eb-f73058e627be admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.507032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae21c-45b7-4ffb-a8eb-f73058e627be admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.507378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae21c-45b7-4ffb-a8eb-f73058e627be admin admin] PATH_INFO: `/v3/projects/e764e4f590fd4e27bdbfb1ed0acb569a/users/0184e23162264b54acb53a063f076380/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.525569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecae21c-45b7-4ffb-a8eb-f73058e627be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e764e4f590fd4e27bdbfb1ed0acb569a, user_id=0184e23162264b54acb53a063f076380, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:56.528462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecae21c-45b7-4ffb-a8eb-f73058e627be admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:56.559617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3d1c2e23a64b849d240f8a7ce9dcc0', 'request_id': 'req-63f62752-d9cd-4b29-9b52-98bd197057ef'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:56.589477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] The token cache is being invalidate because project 7b3d1c2e23a64b849d240f8a7ce9dcc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:00:56.589872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:56.591099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-63f62752-d9cd-4b29-9b52-98bd197057ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:00:56.595206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 935/1883] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:00:56 2022] DELETE /identity/v3/projects/7b3d1c2e23a64b849d240f8a7ce9dcc0 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:56.600961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc560b7-48e4-4026-89bf-f40f5b0c0794 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.601468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc560b7-48e4-4026-89bf-f40f5b0c0794 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.607716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc560b7-48e4-4026-89bf-f40f5b0c0794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.633224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9dc560b7-48e4-4026-89bf-f40f5b0c0794 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:56.635748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9dc560b7-48e4-4026-89bf-f40f5b0c0794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:56.647196 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 949/1884] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:56 2022] PUT /identity/v3/projects/e764e4f590fd4e27bdbfb1ed0acb569a/users/0184e23162264b54acb53a063f076380/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:56.650901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecae21c-45b7-4ffb-a8eb-f73058e627be admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:56.652772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3ecae21c-45b7-4ffb-a8eb-f73058e627be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:56.746232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecae21c-45b7-4ffb-a8eb-f73058e627be admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:56.754878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e5a24-11a4-40cf-af74-63990aedbfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:56.762648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e5a24-11a4-40cf-af74-63990aedbfd4 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.762648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e5a24-11a4-40cf-af74-63990aedbfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.762648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e5a24-11a4-40cf-af74-63990aedbfd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.762648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e5a24-11a4-40cf-af74-63990aedbfd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:56.763190 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 936/1885] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:00:56.763378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e5a24-11a4-40cf-af74-63990aedbfd4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:56.765477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f928f-3450-47e4-936e-ec09a6e5b200 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.766336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f928f-3450-47e4-936e-ec09a6e5b200 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.766620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f928f-3450-47e4-936e-ec09a6e5b200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.774954 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 950/1886] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:56 2022] GET /identity/v3/roles => generated 1427 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:56.779250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e5a24-11a4-40cf-af74-63990aedbfd4 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:56.789907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-192f928f-3450-47e4-936e-ec09a6e5b200 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:56.792042 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e5a24-11a4-40cf-af74-63990aedbfd4 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:56.792725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-192f928f-3450-47e4-936e-ec09a6e5b200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:56.799347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451c8299-4b42-4d5b-a049-9a669f89c88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:56.804391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c8299-4b42-4d5b-a049-9a669f89c88e admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.804391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c8299-4b42-4d5b-a049-9a669f89c88e admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.804391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451c8299-4b42-4d5b-a049-9a669f89c88e admin admin] PATH_INFO: `/v3/projects/e764e4f590fd4e27bdbfb1ed0acb569a/users/0184e23162264b54acb53a063f076380/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.804391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c8299-4b42-4d5b-a049-9a669f89c88e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e764e4f590fd4e27bdbfb1ed0acb569a, user_id=0184e23162264b54acb53a063f076380, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:56.806889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451c8299-4b42-4d5b-a049-9a669f89c88e admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:56.807741 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 937/1887] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:00:56.808714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f928f-3450-47e4-936e-ec09a6e5b200 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:56.820108 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 951/1888] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:56 2022] PUT /identity/v3/projects/e764e4f590fd4e27bdbfb1ed0acb569a/users/0184e23162264b54acb53a063f076380/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:56.820254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f928f-3450-47e4-936e-ec09a6e5b200 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:56.832317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47c86c-64d4-4b7e-818b-faffbac04fc2 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.832604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47c86c-64d4-4b7e-818b-faffbac04fc2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.832881 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47c86c-64d4-4b7e-818b-faffbac04fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.865222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2e47c86c-64d4-4b7e-818b-faffbac04fc2 None None] MFA Rules not processed for user `0184e23162264b54acb53a063f076380`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:56.866566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2e47c86c-64d4-4b7e-818b-faffbac04fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:56.884019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078cf54-e0b2-441c-8109-cb90e842168b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:56.885234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078cf54-e0b2-441c-8109-cb90e842168b admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.885514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078cf54-e0b2-441c-8109-cb90e842168b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.886019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078cf54-e0b2-441c-8109-cb90e842168b admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.886825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078cf54-e0b2-441c-8109-cb90e842168b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:00:56.887319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078cf54-e0b2-441c-8109-cb90e842168b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:56.889043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078cf54-e0b2-441c-8109-cb90e842168b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:56.905008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 938/1889] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:00:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:00:56.909221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078cf54-e0b2-441c-8109-cb90e842168b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:56.922140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078cf54-e0b2-441c-8109-cb90e842168b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:56.936110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2239ab-cc07-4859-8004-88ea8efc78e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:56.938539 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2239ab-cc07-4859-8004-88ea8efc78e3 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:56.940025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2239ab-cc07-4859-8004-88ea8efc78e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:56.940025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2239ab-cc07-4859-8004-88ea8efc78e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:56.941571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2239ab-cc07-4859-8004-88ea8efc78e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:56.947629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2239ab-cc07-4859-8004-88ea8efc78e3 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:57.010628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 952/1890] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:00:57.022811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47c86c-64d4-4b7e-818b-faffbac04fc2 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:57.024714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2e47c86c-64d4-4b7e-818b-faffbac04fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:57.079829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 939/1891] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:56 2022] POST /identity/v3/projects => generated 384 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:57.083649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2239ab-cc07-4859-8004-88ea8efc78e3 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:57.157973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47c86c-64d4-4b7e-818b-faffbac04fc2 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:57.219380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2239ab-cc07-4859-8004-88ea8efc78e3 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:57.227500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c7dd5-97d3-4313-8b3c-30c9d2d5933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:57.228919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c7dd5-97d3-4313-8b3c-30c9d2d5933e admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:57.229331 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c7dd5-97d3-4313-8b3c-30c9d2d5933e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:57.230103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c7dd5-97d3-4313-8b3c-30c9d2d5933e admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:57.231169 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c7dd5-97d3-4313-8b3c-30c9d2d5933e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:57.233298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c7dd5-97d3-4313-8b3c-30c9d2d5933e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:57.239460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f21563-a3de-4bd6-9d38-c3e2e59f65e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:57.240497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f21563-a3de-4bd6-9d38-c3e2e59f65e7 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:57.241225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f21563-a3de-4bd6-9d38-c3e2e59f65e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:57.241503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f21563-a3de-4bd6-9d38-c3e2e59f65e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:57.317304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f21563-a3de-4bd6-9d38-c3e2e59f65e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:57.318925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f21563-a3de-4bd6-9d38-c3e2e59f65e7 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:57.330882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 940/1892] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:57 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:00:57.334742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c7dd5-97d3-4313-8b3c-30c9d2d5933e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:57.346968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 953/1893] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:57.362977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c7dd5-97d3-4313-8b3c-30c9d2d5933e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:57.385691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44188eb5-5a89-445f-bb0b-735b6e6158a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:57.385691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44188eb5-5a89-445f-bb0b-735b6e6158a8 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:57.386456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44188eb5-5a89-445f-bb0b-735b6e6158a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:57.386456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44188eb5-5a89-445f-bb0b-735b6e6158a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:57.386886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44188eb5-5a89-445f-bb0b-735b6e6158a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:57.388806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44188eb5-5a89-445f-bb0b-735b6e6158a8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:57.402497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 941/1894] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:57 2022] GET /identity/v3/roles => generated 1427 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:57.407134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f21563-a3de-4bd6-9d38-c3e2e59f65e7 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:57.417993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f21563-a3de-4bd6-9d38-c3e2e59f65e7 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:57.424761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297ae3c-6d23-4436-b86f-8a2ea02f81df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:57.425147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297ae3c-6d23-4436-b86f-8a2ea02f81df admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:57.425534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297ae3c-6d23-4436-b86f-8a2ea02f81df admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:57.425925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297ae3c-6d23-4436-b86f-8a2ea02f81df admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:57.427068 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297ae3c-6d23-4436-b86f-8a2ea02f81df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:57.428768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297ae3c-6d23-4436-b86f-8a2ea02f81df admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:57.443567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 954/1895] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:57 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:57.444790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44188eb5-5a89-445f-bb0b-735b6e6158a8 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:57.455323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44188eb5-5a89-445f-bb0b-735b6e6158a8 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:57.460229 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a48b2-e9f6-4628-9b16-14593020e005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:57.460977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a48b2-e9f6-4628-9b16-14593020e005 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:57.461255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a48b2-e9f6-4628-9b16-14593020e005 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:57.461555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a48b2-e9f6-4628-9b16-14593020e005 admin admin] PATH_INFO: `/v3/projects/72c38df280044b16a7a714f07751d0d1/users/f445fb8e29c846c9a0e3b98f464a1406/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:57.491842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a48b2-e9f6-4628-9b16-14593020e005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72c38df280044b16a7a714f07751d0d1, user_id=f445fb8e29c846c9a0e3b98f464a1406, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:57.494539 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a48b2-e9f6-4628-9b16-14593020e005 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:57.561981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:57.622246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 942/1896] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:57 2022] PUT /identity/v3/projects/72c38df280044b16a7a714f07751d0d1/users/f445fb8e29c846c9a0e3b98f464a1406/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:00:57.632447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3ba97-6ac2-4d69-8961-edb111e222ce None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:57.632748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3ba97-6ac2-4d69-8961-edb111e222ce None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:57.632994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3ba97-6ac2-4d69-8961-edb111e222ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:57.677181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c2c3ba97-6ac2-4d69-8961-edb111e222ce None None] MFA Rules not processed for user `f445fb8e29c846c9a0e3b98f464a1406`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:57.678440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c2c3ba97-6ac2-4d69-8961-edb111e222ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:57.759970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 955/1897] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:57.765299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a48b2-e9f6-4628-9b16-14593020e005 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:57.831582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3ba97-6ac2-4d69-8961-edb111e222ce None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:57.872074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a48b2-e9f6-4628-9b16-14593020e005 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:57.887401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d5ea9-e0c0-41d3-88a2-d11ab2a1cc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:57.888477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d5ea9-e0c0-41d3-88a2-d11ab2a1cc18 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:57.888923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d5ea9-e0c0-41d3-88a2-d11ab2a1cc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:57.889341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d5ea9-e0c0-41d3-88a2-d11ab2a1cc18 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:57.890155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d5ea9-e0c0-41d3-88a2-d11ab2a1cc18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:57.892366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d5ea9-e0c0-41d3-88a2-d11ab2a1cc18 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:57.938598 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3ba97-6ac2-4d69-8961-edb111e222ce None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:57.943456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a163dc-e698-45cd-baf4-73e84326c1a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:57.943732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a163dc-e698-45cd-baf4-73e84326c1a3 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:57.944261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a163dc-e698-45cd-baf4-73e84326c1a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:57.944261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a163dc-e698-45cd-baf4-73e84326c1a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:57.946143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-59a163dc-e698-45cd-baf4-73e84326c1a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:58.018878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a163dc-e698-45cd-baf4-73e84326c1a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:58.020893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a163dc-e698-45cd-baf4-73e84326c1a3 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:58.039272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 956/1898] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:58.056298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 943/1899] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:57 2022] POST /identity/v3/projects => generated 384 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:00:58.057490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a163dc-e698-45cd-baf4-73e84326c1a3 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:58.076922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a163dc-e698-45cd-baf4-73e84326c1a3 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:58.148676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dae2c-40e8-4540-8820-def1f4512114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:58.149351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dae2c-40e8-4540-8820-def1f4512114 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:58.149651 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dae2c-40e8-4540-8820-def1f4512114 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:58.150063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dae2c-40e8-4540-8820-def1f4512114 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:58.150729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320dae2c-40e8-4540-8820-def1f4512114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:58.152911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320dae2c-40e8-4540-8820-def1f4512114 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:58.313348 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 957/1900] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:58 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:58.318744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d5ea9-e0c0-41d3-88a2-d11ab2a1cc18 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:58.331248 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d5ea9-e0c0-41d3-88a2-d11ab2a1cc18 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:58.336392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898c65c-8719-4d66-bfea-6476bc68384e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:58.337087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898c65c-8719-4d66-bfea-6476bc68384e admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:58.337425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898c65c-8719-4d66-bfea-6476bc68384e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:58.337765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898c65c-8719-4d66-bfea-6476bc68384e admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:58.338529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898c65c-8719-4d66-bfea-6476bc68384e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:58.340249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898c65c-8719-4d66-bfea-6476bc68384e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:58.351604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 944/1901] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:58 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:58.355951 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dae2c-40e8-4540-8820-def1f4512114 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:58.367165 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dae2c-40e8-4540-8820-def1f4512114 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:58.372190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4478356-fadc-4749-94bb-3d3192ee1991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:58.372904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4478356-fadc-4749-94bb-3d3192ee1991 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:58.373303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4478356-fadc-4749-94bb-3d3192ee1991 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:58.373680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4478356-fadc-4749-94bb-3d3192ee1991 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:58.374785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4478356-fadc-4749-94bb-3d3192ee1991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:58.384954 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4478356-fadc-4749-94bb-3d3192ee1991 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:58.401892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 958/1902] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:58 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:58.405857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898c65c-8719-4d66-bfea-6476bc68384e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:58.420263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898c65c-8719-4d66-bfea-6476bc68384e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:58.426811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ce1ec-8af3-4993-8ca0-ba6c6b44f1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:58.428079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ce1ec-8af3-4993-8ca0-ba6c6b44f1b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:58.428079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ce1ec-8af3-4993-8ca0-ba6c6b44f1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:58.428079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ce1ec-8af3-4993-8ca0-ba6c6b44f1b4 admin admin] PATH_INFO: `/v3/projects/87b69aa26d4343c1bd483cc7a30a9668/users/6ff2af2b20464e9f842bb22dfe09a0db/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:58.445119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ce1ec-8af3-4993-8ca0-ba6c6b44f1b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b69aa26d4343c1bd483cc7a30a9668, user_id=6ff2af2b20464e9f842bb22dfe09a0db, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:58.448042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ce1ec-8af3-4993-8ca0-ba6c6b44f1b4 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:58.595428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 945/1903] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:58 2022] PUT /identity/v3/projects/87b69aa26d4343c1bd483cc7a30a9668/users/6ff2af2b20464e9f842bb22dfe09a0db/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:58.618360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fe018-444a-46da-8f4c-3910dc987762 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:58.618851 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fe018-444a-46da-8f4c-3910dc987762 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:58.618931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fe018-444a-46da-8f4c-3910dc987762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:58.681545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-907fe018-444a-46da-8f4c-3910dc987762 None None] MFA Rules not processed for user `6ff2af2b20464e9f842bb22dfe09a0db`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:58.681545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-907fe018-444a-46da-8f4c-3910dc987762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:58.690297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a6ac-7bcc-4cc4-b3bc-27e0364ce092 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:58.690586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a6ac-7bcc-4cc4-b3bc-27e0364ce092 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:58.690879 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31a6ac-7bcc-4cc4-b3bc-27e0364ce092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:58.727917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-4e31a6ac-7bcc-4cc4-b3bc-27e0364ce092 None None] MFA Rules not processed for user `946a7533230b446f8ff34a0a51d4c2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:58.729988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4e31a6ac-7bcc-4cc4-b3bc-27e0364ce092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:58.800632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 959/1904] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:58.805474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fe018-444a-46da-8f4c-3910dc987762 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:58.811424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 946/1905] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:00:58.834471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31a6ac-7bcc-4cc4-b3bc-27e0364ce092 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:58.897488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fe018-444a-46da-8f4c-3910dc987762 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:58.902422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159ac59-bbf9-4163-83f4-119f81db9dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:58.903061 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159ac59-bbf9-4163-83f4-119f81db9dfe admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:58.903387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159ac59-bbf9-4163-83f4-119f81db9dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:58.903665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159ac59-bbf9-4163-83f4-119f81db9dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:58.904292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159ac59-bbf9-4163-83f4-119f81db9dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:58.907452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159ac59-bbf9-4163-83f4-119f81db9dfe admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:58.917279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31a6ac-7bcc-4cc4-b3bc-27e0364ce092 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:58.921507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b8c56-b027-4578-89ed-8964b34f3215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:58.923384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8c56-b027-4578-89ed-8964b34f3215 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:58.923832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8c56-b027-4578-89ed-8964b34f3215 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:58.924259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b8c56-b027-4578-89ed-8964b34f3215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:58.935368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b8c56-b027-4578-89ed-8964b34f3215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:58.936870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b8c56-b027-4578-89ed-8964b34f3215 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:58.946276 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 947/1906] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:00:58.987413 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:00:59.036194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b8c56-b027-4578-89ed-8964b34f3215 service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:59.036759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 960/1907] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:00:58 2022] POST /identity/v3/projects => generated 386 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:00:59.054201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b8c56-b027-4578-89ed-8964b34f3215 service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:59.122520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38099e-99d0-46cb-b1d2-770b86aeea4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:59.123180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38099e-99d0-46cb-b1d2-770b86aeea4c admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:59.123462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38099e-99d0-46cb-b1d2-770b86aeea4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:59.123738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38099e-99d0-46cb-b1d2-770b86aeea4c admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:59.124365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38099e-99d0-46cb-b1d2-770b86aeea4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:59.127088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38099e-99d0-46cb-b1d2-770b86aeea4c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:59.310687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 948/1908] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:00:59 2022] POST /identity/v3/users => generated 300 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:00:59.312169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159ac59-bbf9-4163-83f4-119f81db9dfe admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:59.323631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159ac59-bbf9-4163-83f4-119f81db9dfe admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:59.328377 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b71b6f-00f5-4aea-ab30-9eb8a503a6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:59.328933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b71b6f-00f5-4aea-ab30-9eb8a503a6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:59.329215 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b71b6f-00f5-4aea-ab30-9eb8a503a6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:59.329489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b71b6f-00f5-4aea-ab30-9eb8a503a6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:59.330133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b71b6f-00f5-4aea-ab30-9eb8a503a6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:59.331941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b71b6f-00f5-4aea-ab30-9eb8a503a6c0 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:59.341942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 961/1909] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:59 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:00:59.346259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38099e-99d0-46cb-b1d2-770b86aeea4c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:59.356359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38099e-99d0-46cb-b1d2-770b86aeea4c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:59.361530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4565e-007d-4278-84fb-e74388da5960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:59.362177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4565e-007d-4278-84fb-e74388da5960 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:59.362461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4565e-007d-4278-84fb-e74388da5960 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:59.362899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4565e-007d-4278-84fb-e74388da5960 admin admin] PATH_INFO: `/v3/projects/d2e86a67b40149c3b04af400b289908a/users/ed36203d3de94b39b52eff92a9d6208b/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:59.378127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4565e-007d-4278-84fb-e74388da5960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e86a67b40149c3b04af400b289908a, user_id=ed36203d3de94b39b52eff92a9d6208b, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:59.380884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4565e-007d-4278-84fb-e74388da5960 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:59.518817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 949/1910] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:59 2022] PUT /identity/v3/projects/d2e86a67b40149c3b04af400b289908a/users/ed36203d3de94b39b52eff92a9d6208b/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:59.528639 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b71b6f-00f5-4aea-ab30-9eb8a503a6c0 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:59.539361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b71b6f-00f5-4aea-ab30-9eb8a503a6c0 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:59.599294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab71d2-e089-429a-ab1a-9b21e70c5196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:59.599898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab71d2-e089-429a-ab1a-9b21e70c5196 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:59.600172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab71d2-e089-429a-ab1a-9b21e70c5196 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:59.600438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab71d2-e089-429a-ab1a-9b21e70c5196 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:59.601073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab71d2-e089-429a-ab1a-9b21e70c5196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:59.602699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab71d2-e089-429a-ab1a-9b21e70c5196 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:59.613134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 962/1911] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:00:59 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:00:59.617239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4565e-007d-4278-84fb-e74388da5960 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:59.627890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4565e-007d-4278-84fb-e74388da5960 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:59.632584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de67c0-4fee-43ee-b270-22454aa85cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:59.636143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de67c0-4fee-43ee-b270-22454aa85cab admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:59.636143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de67c0-4fee-43ee-b270-22454aa85cab admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:59.636143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de67c0-4fee-43ee-b270-22454aa85cab admin admin] PATH_INFO: `/v3/projects/d2e86a67b40149c3b04af400b289908a/users/ed36203d3de94b39b52eff92a9d6208b/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:59.636143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de67c0-4fee-43ee-b270-22454aa85cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2e86a67b40149c3b04af400b289908a, user_id=ed36203d3de94b39b52eff92a9d6208b, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:59.638311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de67c0-4fee-43ee-b270-22454aa85cab admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:59.648692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 950/1912] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:00:59 2022] PUT /identity/v3/projects/d2e86a67b40149c3b04af400b289908a/users/ed36203d3de94b39b52eff92a9d6208b/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:00:59.659343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690d48f-5153-43be-95e0-b593c414ba78 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:59.659835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690d48f-5153-43be-95e0-b593c414ba78 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:59.659927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7690d48f-5153-43be-95e0-b593c414ba78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:59.688232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-7690d48f-5153-43be-95e0-b593c414ba78 None None] MFA Rules not processed for user `ed36203d3de94b39b52eff92a9d6208b`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:00:59.689484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-7690d48f-5153-43be-95e0-b593c414ba78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:00:59.761882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 963/1913] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:00:59 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:00:59.831708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de67c0-4fee-43ee-b270-22454aa85cab admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:00:59.888761 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de67c0-4fee-43ee-b270-22454aa85cab admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:00:59.892549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c48b9-c00d-4ed8-9f86-99933dca24c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:00:59.893161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c48b9-c00d-4ed8-9f86-99933dca24c0 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:00:59.893426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c48b9-c00d-4ed8-9f86-99933dca24c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:00:59.893695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c48b9-c00d-4ed8-9f86-99933dca24c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:00:59.954752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c48b9-c00d-4ed8-9f86-99933dca24c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:00:59.956232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c48b9-c00d-4ed8-9f86-99933dca24c0 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:00:59.965393 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 951/1914] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:00:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:00.700814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:01.112901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:01.497000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d4128-c2df-40db-908d-0a6e0d1d2f24 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:01.497208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d4128-c2df-40db-908d-0a6e0d1d2f24 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:01.497486 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d4128-c2df-40db-908d-0a6e0d1d2f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:01.517392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ee2d4128-c2df-40db-908d-0a6e0d1d2f24 None None] MFA Rules not processed for user `946a7533230b446f8ff34a0a51d4c2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:01.518594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ee2d4128-c2df-40db-908d-0a6e0d1d2f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:01.576633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 964/1915] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:01.998211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:02.398723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:02.867150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:03.141294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c48b9-c00d-4ed8-9f86-99933dca24c0 service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:03.143576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-043c48b9-c00d-4ed8-9f86-99933dca24c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:03.172247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c48b9-c00d-4ed8-9f86-99933dca24c0 service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:03.179911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf1d9d-984e-4ad4-9304-1abbc4744399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:03.185884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf1d9d-984e-4ad4-9304-1abbc4744399 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:03.185884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf1d9d-984e-4ad4-9304-1abbc4744399 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:03.185884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf1d9d-984e-4ad4-9304-1abbc4744399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:03.244338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf1d9d-984e-4ad4-9304-1abbc4744399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:03.254870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf1d9d-984e-4ad4-9304-1abbc4744399 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:03.256290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 952/1916] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:03.552988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:03.867163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:04.107407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:04.518396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:04.718073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:06.490400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d4128-c2df-40db-908d-0a6e0d1d2f24 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:06.499825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d4128-c2df-40db-908d-0a6e0d1d2f24 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:06.503374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9839f-1132-4239-9518-7ed6e24e5568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:06.503961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9839f-1132-4239-9518-7ed6e24e5568 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:06.504236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9839f-1132-4239-9518-7ed6e24e5568 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:06.504501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9839f-1132-4239-9518-7ed6e24e5568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:06.513475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9839f-1132-4239-9518-7ed6e24e5568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:06.514894 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9839f-1132-4239-9518-7ed6e24e5568 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:06.525108 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 965/1917] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:06 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:14.333102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf1d9d-984e-4ad4-9304-1abbc4744399 service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:14.399373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf1d9d-984e-4ad4-9304-1abbc4744399 service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:14.403904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:14.404529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:14.404804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:14.405072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] PATH_INFO: `/v3/users/c6c5c6109dda43a0a96ed15a9e535bbc` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:14.406198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c5c6109dda43a0a96ed15a9e535bbc)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:14.408927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:14.411443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:14.731520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c5c6109dda43a0a96ed15a9e535bbc'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:14.732177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c5c6109dda43a0a96ed15a9e535bbc'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:14.732827 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c5c6109dda43a0a96ed15a9e535bbc'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:14.747179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9839f-1132-4239-9518-7ed6e24e5568 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:14.758590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9839f-1132-4239-9518-7ed6e24e5568 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:14.763167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:14.763785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:14.764066 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:14.764338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] PATH_INFO: `/v3/users/f445fb8e29c846c9a0e3b98f464a1406` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:14.765459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f445fb8e29c846c9a0e3b98f464a1406)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:14.768047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:14.848284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c5c6109dda43a0a96ed15a9e535bbc'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:14.861572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43b7629a-70f0-4504-a73f-b7de77ba9344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c5c6109dda43a0a96ed15a9e535bbc'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:14.874228 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 953/1918] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:14 2022] DELETE /identity/v3/users/c6c5c6109dda43a0a96ed15a9e535bbc => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:14.879147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaaaab-e602-418f-a092-038c7c3f31ea None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:14.879436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaaaab-e602-418f-a092-038c7c3f31ea None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:14.879711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eaaaab-e602-418f-a092-038c7c3f31ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:14.901776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-f8eaaaab-e602-418f-a092-038c7c3f31ea None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:14.902989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f8eaaaab-e602-418f-a092-038c7c3f31ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:14.966936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 954/1919] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:14.977488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaaaab-e602-418f-a092-038c7c3f31ea None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:14.978853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f8eaaaab-e602-418f-a092-038c7c3f31ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:15.011407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f445fb8e29c846c9a0e3b98f464a1406'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:15.012056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f445fb8e29c846c9a0e3b98f464a1406'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:15.012655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f445fb8e29c846c9a0e3b98f464a1406'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:15.047270 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eaaaab-e602-418f-a092-038c7c3f31ea None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:15.052148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45471df-a674-4f60-8f03-dd1e49693154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:15.052743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45471df-a674-4f60-8f03-dd1e49693154 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:15.053018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45471df-a674-4f60-8f03-dd1e49693154 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:15.053287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45471df-a674-4f60-8f03-dd1e49693154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:15.054972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c45471df-a674-4f60-8f03-dd1e49693154 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:15.066991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45471df-a674-4f60-8f03-dd1e49693154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:15.068358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45471df-a674-4f60-8f03-dd1e49693154 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:15.077643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 955/1920] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:15.117420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f445fb8e29c846c9a0e3b98f464a1406'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:15.130982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f445fb8e29c846c9a0e3b98f464a1406'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:15.144816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 966/1921] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:14 2022] DELETE /identity/v3/users/f445fb8e29c846c9a0e3b98f464a1406 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:15.149295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dfb87-c0d5-4d0d-a61f-f5f378d5a164 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:15.149582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dfb87-c0d5-4d0d-a61f-f5f378d5a164 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:15.149963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1dfb87-c0d5-4d0d-a61f-f5f378d5a164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:15.170380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-4c1dfb87-c0d5-4d0d-a61f-f5f378d5a164 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:15.172040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4c1dfb87-c0d5-4d0d-a61f-f5f378d5a164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:15.190599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:15.232555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 956/1922] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:15.242393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:15.243808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:15.301964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264ece9-290b-407c-919e-7ce5e9b39c4c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:15.305581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d01faa-a4af-481d-bb8b-3b3284c5765e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:15.306180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d01faa-a4af-481d-bb8b-3b3284c5765e service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:15.306461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d01faa-a4af-481d-bb8b-3b3284c5765e service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:15.306727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d01faa-a4af-481d-bb8b-3b3284c5765e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:15.316211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d01faa-a4af-481d-bb8b-3b3284c5765e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:15.317637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d01faa-a4af-481d-bb8b-3b3284c5765e service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:15.327818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 967/1923] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:15.597208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dfb87-c0d5-4d0d-a61f-f5f378d5a164 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:15.598997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4c1dfb87-c0d5-4d0d-a61f-f5f378d5a164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:15.613402 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1dfb87-c0d5-4d0d-a61f-f5f378d5a164 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:15.618600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:15.619514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:15.619792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:15.623167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] PATH_INFO: `/v3/projects/773d0469a77a47efb7563d3a83884d4d` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:15.623167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=773d0469a77a47efb7563d3a83884d4d)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:15.623514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:15.791150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773d0469a77a47efb7563d3a83884d4d', 'request_id': 'req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:15.797438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773d0469a77a47efb7563d3a83884d4d', 'request_id': 'req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:15.904132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d01faa-a4af-481d-bb8b-3b3284c5765e service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:15.904132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-33d01faa-a4af-481d-bb8b-3b3284c5765e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:15.912499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d01faa-a4af-481d-bb8b-3b3284c5765e service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:15.924477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773d0469a77a47efb7563d3a83884d4d', 'request_id': 'req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:15.926766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:15.938800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] The token cache is being invalidate because project 773d0469a77a47efb7563d3a83884d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:15.939213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:15.940484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:15.942273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 957/1924] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:15 2022] DELETE /identity/v3/projects/773d0469a77a47efb7563d3a83884d4d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:15.945950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:15.948286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:15.984072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:15.984720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:15.985001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:15.985274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] PATH_INFO: `/v3/projects/72c38df280044b16a7a714f07751d0d1` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:15.986742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72c38df280044b16a7a714f07751d0d1)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:15.988903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:16.044309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb8d7f-0e07-4bd0-9198-e00efeabc38f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:16.044309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:16.044309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:16.044309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:16.044906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] PATH_INFO: `/v3/users/0184e23162264b54acb53a063f076380` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:16.044906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0184e23162264b54acb53a063f076380)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:16.044906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:16.124212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72c38df280044b16a7a714f07751d0d1', 'request_id': 'req-30cd59c9-74de-4dbf-8472-24eee12c1cc9'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.132771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72c38df280044b16a7a714f07751d0d1', 'request_id': 'req-30cd59c9-74de-4dbf-8472-24eee12c1cc9'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.141857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72c38df280044b16a7a714f07751d0d1', 'request_id': 'req-30cd59c9-74de-4dbf-8472-24eee12c1cc9'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.248065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] The token cache is being invalidate because project 72c38df280044b16a7a714f07751d0d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:16.248355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.248976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.250494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 968/1925] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:15 2022] DELETE /identity/v3/projects/72c38df280044b16a7a714f07751d0d1 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:16.255456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:16.256957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:16.259875 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0184e23162264b54acb53a063f076380'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.260567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0184e23162264b54acb53a063f076380'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.261167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0184e23162264b54acb53a063f076380'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.262851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:16.322119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd59c9-74de-4dbf-8472-24eee12c1cc9 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:16.326828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:16.327488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:16.327768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:16.328043 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] PATH_INFO: `/v3/users/6ff2af2b20464e9f842bb22dfe09a0db` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:16.329159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff2af2b20464e9f842bb22dfe09a0db)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:16.331125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:16.351577 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0184e23162264b54acb53a063f076380'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.366334 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0184e23162264b54acb53a063f076380'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.379059 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 958/1926] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:15 2022] DELETE /identity/v3/users/0184e23162264b54acb53a063f076380 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:16.542034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff2af2b20464e9f842bb22dfe09a0db'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.542751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff2af2b20464e9f842bb22dfe09a0db'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.543372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff2af2b20464e9f842bb22dfe09a0db'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.557280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:16.620080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff2af2b20464e9f842bb22dfe09a0db'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.633330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff2af2b20464e9f842bb22dfe09a0db'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:16.646622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 969/1927] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:16 2022] DELETE /identity/v3/users/6ff2af2b20464e9f842bb22dfe09a0db => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:16.822397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:16.823898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:16.912801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d230cd-a405-43ff-b31b-f8cc9bc53593 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:16.978942 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:16.979619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:16.979942 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:16.980186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] PATH_INFO: `/v3/projects/e764e4f590fd4e27bdbfb1ed0acb569a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:16.981341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e764e4f590fd4e27bdbfb1ed0acb569a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:16.983300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:17.106806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e764e4f590fd4e27bdbfb1ed0acb569a', 'request_id': 'req-44137470-8adb-42a0-94a0-f2d6485fbf14'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.114381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e764e4f590fd4e27bdbfb1ed0acb569a', 'request_id': 'req-44137470-8adb-42a0-94a0-f2d6485fbf14'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.124776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:17.126694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:17.170134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd9f2e-1513-4438-a62f-7aeca08add9c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:17.424656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:17.424656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:17.424656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:17.424656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] PATH_INFO: `/v3/projects/87b69aa26d4343c1bd483cc7a30a9668` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:17.426743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b69aa26d4343c1bd483cc7a30a9668)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:17.426743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:17.427263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e764e4f590fd4e27bdbfb1ed0acb569a', 'request_id': 'req-44137470-8adb-42a0-94a0-f2d6485fbf14'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.427263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] The token cache is being invalidate because project e764e4f590fd4e27bdbfb1ed0acb569a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:17.427263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.427263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-44137470-8adb-42a0-94a0-f2d6485fbf14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.427263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 959/1928] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:16 2022] DELETE /identity/v3/projects/e764e4f590fd4e27bdbfb1ed0acb569a => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:17.427921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5492ac-5a54-4616-a6c6-9191ccd3482b None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:17.427921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5492ac-5a54-4616-a6c6-9191ccd3482b None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:17.427921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5492ac-5a54-4616-a6c6-9191ccd3482b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:17.427921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:17.427921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-3a5492ac-5a54-4616-a6c6-9191ccd3482b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:17.427921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3a5492ac-5a54-4616-a6c6-9191ccd3482b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:17.428483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 960/1929] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:17.428483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9df83-74f7-4041-b788-47d791f70f91 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:17.428483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9df83-74f7-4041-b788-47d791f70f91 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:17.428483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9df83-74f7-4041-b788-47d791f70f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:17.428483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-acb9df83-74f7-4041-b788-47d791f70f91 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:17.428483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-acb9df83-74f7-4041-b788-47d791f70f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:17.428944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 961/1930] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:17.428944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9df83-74f7-4041-b788-47d791f70f91 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:17.428944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9df83-74f7-4041-b788-47d791f70f91 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:17.428944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78032af6-5407-4b9f-acee-a5b64ec5f474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:17.429269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78032af6-5407-4b9f-acee-a5b64ec5f474 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:17.429269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78032af6-5407-4b9f-acee-a5b64ec5f474 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:17.429269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78032af6-5407-4b9f-acee-a5b64ec5f474 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:17.429269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78032af6-5407-4b9f-acee-a5b64ec5f474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:01:17.429269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78032af6-5407-4b9f-acee-a5b64ec5f474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:17.429269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78032af6-5407-4b9f-acee-a5b64ec5f474 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:17.429738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 962/1931] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:01:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:01:17.429738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78032af6-5407-4b9f-acee-a5b64ec5f474 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:17.434901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78032af6-5407-4b9f-acee-a5b64ec5f474 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:17.440000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87102eac-0c45-4bf7-b046-bcb0a51590ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:17.440582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87102eac-0c45-4bf7-b046-bcb0a51590ef admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:17.440909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87102eac-0c45-4bf7-b046-bcb0a51590ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:17.441182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87102eac-0c45-4bf7-b046-bcb0a51590ef admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:17.441800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87102eac-0c45-4bf7-b046-bcb0a51590ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:17.443752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87102eac-0c45-4bf7-b046-bcb0a51590ef admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:17.495903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b69aa26d4343c1bd483cc7a30a9668', 'request_id': 'req-4536c1dd-a82b-4157-a5a1-99216746525d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.502304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b69aa26d4343c1bd483cc7a30a9668', 'request_id': 'req-4536c1dd-a82b-4157-a5a1-99216746525d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.508146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b69aa26d4343c1bd483cc7a30a9668', 'request_id': 'req-4536c1dd-a82b-4157-a5a1-99216746525d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.575455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 963/1932] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:17 2022] POST /identity/v3/projects => generated 368 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:17.580400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87102eac-0c45-4bf7-b046-bcb0a51590ef admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:17.605019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] The token cache is being invalidate because project 87b69aa26d4343c1bd483cc7a30a9668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:17.605308 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.605981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.607546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 970/1933] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:17 2022] DELETE /identity/v3/projects/87b69aa26d4343c1bd483cc7a30a9668 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:17.611328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:17.612983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:17.641860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87102eac-0c45-4bf7-b046-bcb0a51590ef admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:17.647872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651398af-e2c3-4546-ac3a-a1b48d063c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:17.648484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651398af-e2c3-4546-ac3a-a1b48d063c7d admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:17.648760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651398af-e2c3-4546-ac3a-a1b48d063c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:17.649032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651398af-e2c3-4546-ac3a-a1b48d063c7d admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:17.649657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651398af-e2c3-4546-ac3a-a1b48d063c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:17.651620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651398af-e2c3-4546-ac3a-a1b48d063c7d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:17.673734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:17.678859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536c1dd-a82b-4157-a5a1-99216746525d admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:17.683544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:17.684173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:17.684452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:17.684723 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] PATH_INFO: `/v3/users/ed36203d3de94b39b52eff92a9d6208b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:17.685858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed36203d3de94b39b52eff92a9d6208b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:17.687814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:17.809000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 964/1934] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:17 2022] POST /identity/v3/users => generated 285 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:17.812966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651398af-e2c3-4546-ac3a-a1b48d063c7d admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:17.815124 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-651398af-e2c3-4546-ac3a-a1b48d063c7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:17.826903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651398af-e2c3-4546-ac3a-a1b48d063c7d admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:17.832271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4044c-0040-4d5c-b2ab-b6f019e394f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:17.832874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4044c-0040-4d5c-b2ab-b6f019e394f4 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:17.833152 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4044c-0040-4d5c-b2ab-b6f019e394f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:17.833423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4044c-0040-4d5c-b2ab-b6f019e394f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:17.834047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4044c-0040-4d5c-b2ab-b6f019e394f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:17.835645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4044c-0040-4d5c-b2ab-b6f019e394f4 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:17.845958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 965/1935] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:17 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:01:17.849514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4044c-0040-4d5c-b2ab-b6f019e394f4 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:17.858832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4044c-0040-4d5c-b2ab-b6f019e394f4 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:17.863534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9db443-8d6d-4501-982d-b7e4d2aaafad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:17.864093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9db443-8d6d-4501-982d-b7e4d2aaafad admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:17.864369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9db443-8d6d-4501-982d-b7e4d2aaafad admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:17.864639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9db443-8d6d-4501-982d-b7e4d2aaafad admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:17.865245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9db443-8d6d-4501-982d-b7e4d2aaafad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:17.866787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9db443-8d6d-4501-982d-b7e4d2aaafad admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:17.877378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 966/1936] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:17 2022] GET /identity/v3/roles => generated 1427 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:01:17.881645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9db443-8d6d-4501-982d-b7e4d2aaafad admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:17.891180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9db443-8d6d-4501-982d-b7e4d2aaafad admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:17.896496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2a983-dd67-4e70-84e2-86a7e8574868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:17.897127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2a983-dd67-4e70-84e2-86a7e8574868 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:17.897404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2a983-dd67-4e70-84e2-86a7e8574868 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:17.897677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2a983-dd67-4e70-84e2-86a7e8574868 admin admin] PATH_INFO: `/v3/projects/8285a764591941dfaa26ac5a9d1a5b1b/users/ae357b6df430406e9838c10f12d0311a/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:17.912107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2a983-dd67-4e70-84e2-86a7e8574868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8285a764591941dfaa26ac5a9d1a5b1b, user_id=ae357b6df430406e9838c10f12d0311a, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:17.914717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2a983-dd67-4e70-84e2-86a7e8574868 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:17.977131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed36203d3de94b39b52eff92a9d6208b'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.977901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed36203d3de94b39b52eff92a9d6208b'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.978534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed36203d3de94b39b52eff92a9d6208b'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:17.998004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 967/1937] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:17 2022] PUT /identity/v3/projects/8285a764591941dfaa26ac5a9d1a5b1b/users/ae357b6df430406e9838c10f12d0311a/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:18.010026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f773a-4af5-476e-bd15-c05c9d1b64de None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:18.010239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f773a-4af5-476e-bd15-c05c9d1b64de None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:18.010519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f773a-4af5-476e-bd15-c05c9d1b64de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:18.038572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-770f773a-4af5-476e-bd15-c05c9d1b64de None None] MFA Rules not processed for user `ae357b6df430406e9838c10f12d0311a`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:18.039824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-770f773a-4af5-476e-bd15-c05c9d1b64de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:18.094318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed36203d3de94b39b52eff92a9d6208b'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:18.106365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 968/1938] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:18.107863 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed36203d3de94b39b52eff92a9d6208b'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:18.111876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f773a-4af5-476e-bd15-c05c9d1b64de None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:18.113211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-770f773a-4af5-476e-bd15-c05c9d1b64de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:18.120674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 971/1939] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:17 2022] DELETE /identity/v3/users/ed36203d3de94b39b52eff92a9d6208b => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:18.198782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f773a-4af5-476e-bd15-c05c9d1b64de None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:18.275742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8d3d5-11dd-4bac-8fea-de73ea348d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:18.276661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8d3d5-11dd-4bac-8fea-de73ea348d09 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:18.277327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8d3d5-11dd-4bac-8fea-de73ea348d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:18.277812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a8d3d5-11dd-4bac-8fea-de73ea348d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:18.278752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8d3d5-11dd-4bac-8fea-de73ea348d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:18.282268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a8d3d5-11dd-4bac-8fea-de73ea348d09 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:18.399414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 969/1940] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:18 2022] POST /identity/v3/projects => generated 370 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:18.403710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:18.415794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81b03e-6e54-4a45-b0a5-761e9ba8d9c9 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:18.481091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c098ae2-7754-4e96-b089-facc1c6040f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:18.481707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c098ae2-7754-4e96-b089-facc1c6040f8 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:18.482186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c098ae2-7754-4e96-b089-facc1c6040f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:18.482459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c098ae2-7754-4e96-b089-facc1c6040f8 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:18.483122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c098ae2-7754-4e96-b089-facc1c6040f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:18.485053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c098ae2-7754-4e96-b089-facc1c6040f8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:18.517771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:18.631491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 972/1941] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:18 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:18.635479 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8d3d5-11dd-4bac-8fea-de73ea348d09 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:18.646088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a8d3d5-11dd-4bac-8fea-de73ea348d09 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:18.650675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8773bb-c4b8-451d-b59b-ba1c38267c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:18.651300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8773bb-c4b8-451d-b59b-ba1c38267c42 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:18.651598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8773bb-c4b8-451d-b59b-ba1c38267c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:18.652049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8773bb-c4b8-451d-b59b-ba1c38267c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:18.652581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8773bb-c4b8-451d-b59b-ba1c38267c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:18.654157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8773bb-c4b8-451d-b59b-ba1c38267c42 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:18.665474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 970/1942] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:18 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:18.669584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c098ae2-7754-4e96-b089-facc1c6040f8 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:18.681253 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c098ae2-7754-4e96-b089-facc1c6040f8 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:18.687094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf8c9d-7d3b-4645-ad3d-fea05996e2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:18.687094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf8c9d-7d3b-4645-ad3d-fea05996e2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:18.687527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf8c9d-7d3b-4645-ad3d-fea05996e2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:18.687877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf8c9d-7d3b-4645-ad3d-fea05996e2a7 admin admin] PATH_INFO: `/v3/projects/faa822654d0445f796c78b170fbce9ae/users/af0881cb1e31492dae24947ac7f4c226/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:18.688425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8773bb-c4b8-451d-b59b-ba1c38267c42 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:18.691345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7e8773bb-c4b8-451d-b59b-ba1c38267c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:18.706300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf8c9d-7d3b-4645-ad3d-fea05996e2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa822654d0445f796c78b170fbce9ae, user_id=af0881cb1e31492dae24947ac7f4c226, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:18.709002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf8c9d-7d3b-4645-ad3d-fea05996e2a7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:18.709787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8773bb-c4b8-451d-b59b-ba1c38267c42 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:18.714752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:18.715884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:18.716170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:18.716441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] PATH_INFO: `/v3/projects/d2e86a67b40149c3b04af400b289908a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:18.717554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2e86a67b40149c3b04af400b289908a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:18.719496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:18.785843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 973/1943] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:18 2022] PUT /identity/v3/projects/faa822654d0445f796c78b170fbce9ae/users/af0881cb1e31492dae24947ac7f4c226/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:18.789670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf8c9d-7d3b-4645-ad3d-fea05996e2a7 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:18.832488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:18.832733 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2e86a67b40149c3b04af400b289908a', 'request_id': 'req-d6514b14-7978-4246-b413-c4eb27393adf'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:18.839220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2e86a67b40149c3b04af400b289908a', 'request_id': 'req-d6514b14-7978-4246-b413-c4eb27393adf'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:18.852032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf8c9d-7d3b-4645-ad3d-fea05996e2a7 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:18.911090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8286ee-53d3-4bd4-bfe0-f499bc5a02f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:18.911724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8286ee-53d3-4bd4-bfe0-f499bc5a02f6 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:18.912007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8286ee-53d3-4bd4-bfe0-f499bc5a02f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:18.912279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8286ee-53d3-4bd4-bfe0-f499bc5a02f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:18.913058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8286ee-53d3-4bd4-bfe0-f499bc5a02f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:18.914643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8286ee-53d3-4bd4-bfe0-f499bc5a02f6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:18.925033 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 974/1944] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:18 2022] GET /identity/v3/roles => generated 1427 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:01:18.929032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8286ee-53d3-4bd4-bfe0-f499bc5a02f6 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:18.938712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8286ee-53d3-4bd4-bfe0-f499bc5a02f6 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:18.944094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f2861-af03-4482-b67a-394e06a05ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:18.944663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f2861-af03-4482-b67a-394e06a05ada admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:18.944940 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f2861-af03-4482-b67a-394e06a05ada admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:18.945214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f2861-af03-4482-b67a-394e06a05ada admin admin] PATH_INFO: `/v3/projects/faa822654d0445f796c78b170fbce9ae/users/af0881cb1e31492dae24947ac7f4c226/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:18.947179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f2861-af03-4482-b67a-394e06a05ada admin admin] RBAC: Authorizing `identity:create_grant(project_id=faa822654d0445f796c78b170fbce9ae, user_id=af0881cb1e31492dae24947ac7f4c226, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:18.949874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f2861-af03-4482-b67a-394e06a05ada admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:18.960323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 975/1945] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:18 2022] PUT /identity/v3/projects/faa822654d0445f796c78b170fbce9ae/users/af0881cb1e31492dae24947ac7f4c226/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:18.970376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03406d8f-5503-4fdb-a7a3-346a079b93e6 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:18.970723 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03406d8f-5503-4fdb-a7a3-346a079b93e6 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:18.971011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03406d8f-5503-4fdb-a7a3-346a079b93e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:18.991544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2e86a67b40149c3b04af400b289908a', 'request_id': 'req-d6514b14-7978-4246-b413-c4eb27393adf'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:19.003786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-03406d8f-5503-4fdb-a7a3-346a079b93e6 None None] MFA Rules not processed for user `af0881cb1e31492dae24947ac7f4c226`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:19.005816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-03406d8f-5503-4fdb-a7a3-346a079b93e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:19.008511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] The token cache is being invalidate because project d2e86a67b40149c3b04af400b289908a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:19.008980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:19.009946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d6514b14-7978-4246-b413-c4eb27393adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:19.012329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 971/1946] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:18 2022] DELETE /identity/v3/projects/d2e86a67b40149c3b04af400b289908a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:19.025403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b08c1c-303a-4779-bc21-21244cfc546c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.025709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b08c1c-303a-4779-bc21-21244cfc546c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.025984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b08c1c-303a-4779-bc21-21244cfc546c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.045405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-46b08c1c-303a-4779-bc21-21244cfc546c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:19.046594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-46b08c1c-303a-4779-bc21-21244cfc546c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:19.077550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 976/1947] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:19.078461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:19.089799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03406d8f-5503-4fdb-a7a3-346a079b93e6 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:19.091518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-03406d8f-5503-4fdb-a7a3-346a079b93e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:19.113759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 972/1948] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:19.122386 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7be965-71e5-4648-904d-46598623c6ab None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.123257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7be965-71e5-4648-904d-46598623c6ab None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.123643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7be965-71e5-4648-904d-46598623c6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.153948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-aa7be965-71e5-4648-904d-46598623c6ab None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:19.155390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-aa7be965-71e5-4648-904d-46598623c6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:19.158712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03406d8f-5503-4fdb-a7a3-346a079b93e6 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:19.163787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eda09-2c45-4f81-9c5f-a15f0f0ea22f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:19.167070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eda09-2c45-4f81-9c5f-a15f0f0ea22f service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.167222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 973/1949] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:19.167317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eda09-2c45-4f81-9c5f-a15f0f0ea22f service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.167317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9eda09-2c45-4f81-9c5f-a15f0f0ea22f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.168619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7be965-71e5-4648-904d-46598623c6ab None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:19.182946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7be965-71e5-4648-904d-46598623c6ab None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:19.184967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eda09-2c45-4f81-9c5f-a15f0f0ea22f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:19.186689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9eda09-2c45-4f81-9c5f-a15f0f0ea22f service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:19.204154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d2393-c302-4fea-bb3a-147c18bd3719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:19.204154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d2393-c302-4fea-bb3a-147c18bd3719 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.204154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d2393-c302-4fea-bb3a-147c18bd3719 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.204154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d2393-c302-4fea-bb3a-147c18bd3719 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.206094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d2393-c302-4fea-bb3a-147c18bd3719 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:01:19.206094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d2393-c302-4fea-bb3a-147c18bd3719 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:19.206094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d2393-c302-4fea-bb3a-147c18bd3719 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:19.206332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 977/1950] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:19.221620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 974/1951] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:01:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:01:19.225079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eda09-2c45-4f81-9c5f-a15f0f0ea22f service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:19.239373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9eda09-2c45-4f81-9c5f-a15f0f0ea22f service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:19.244407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199e8b9-92e8-4580-b5f9-e69b5b137370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:19.245039 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199e8b9-92e8-4580-b5f9-e69b5b137370 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.245311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199e8b9-92e8-4580-b5f9-e69b5b137370 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.245686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199e8b9-92e8-4580-b5f9-e69b5b137370 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.246345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199e8b9-92e8-4580-b5f9-e69b5b137370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:19.248326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199e8b9-92e8-4580-b5f9-e69b5b137370 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:19.404482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d2393-c302-4fea-bb3a-147c18bd3719 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:19.415654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d2393-c302-4fea-bb3a-147c18bd3719 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:19.419617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fe870-b5c6-4fd6-a6b5-0dad93dfd8e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:19.420293 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fe870-b5c6-4fd6-a6b5-0dad93dfd8e5 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.420633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fe870-b5c6-4fd6-a6b5-0dad93dfd8e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.420970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fe870-b5c6-4fd6-a6b5-0dad93dfd8e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.422760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-207fe870-b5c6-4fd6-a6b5-0dad93dfd8e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:19.433678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 978/1952] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:19 2022] POST /identity/v3/projects => generated 374 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:19.438392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199e8b9-92e8-4580-b5f9-e69b5b137370 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:19.487334 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fe870-b5c6-4fd6-a6b5-0dad93dfd8e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:19.488757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fe870-b5c6-4fd6-a6b5-0dad93dfd8e5 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:19.498207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 975/1953] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:19.525505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199e8b9-92e8-4580-b5f9-e69b5b137370 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:19.531102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3534051-762d-46c4-9a03-a8f2f77a2307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:19.531767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3534051-762d-46c4-9a03-a8f2f77a2307 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.534123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3534051-762d-46c4-9a03-a8f2f77a2307 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.534123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3534051-762d-46c4-9a03-a8f2f77a2307 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.534123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3534051-762d-46c4-9a03-a8f2f77a2307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:19.535545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3534051-762d-46c4-9a03-a8f2f77a2307 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:19.575718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:19.660082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 979/1954] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:19 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:19.664189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fe870-b5c6-4fd6-a6b5-0dad93dfd8e5 service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:19.675947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fe870-b5c6-4fd6-a6b5-0dad93dfd8e5 service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:19.680662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfd2d9-ce30-4b1b-96a4-9bd9d4b440a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:19.681361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfd2d9-ce30-4b1b-96a4-9bd9d4b440a5 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.681790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfd2d9-ce30-4b1b-96a4-9bd9d4b440a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.682066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfd2d9-ce30-4b1b-96a4-9bd9d4b440a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.682735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedfd2d9-ce30-4b1b-96a4-9bd9d4b440a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:19.684378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedfd2d9-ce30-4b1b-96a4-9bd9d4b440a5 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:19.695241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 976/1955] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:19 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:01:19.699306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3534051-762d-46c4-9a03-a8f2f77a2307 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:19.709516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3534051-762d-46c4-9a03-a8f2f77a2307 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:19.714186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461f5fa-def2-4b20-9317-750e47b6abbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:19.714793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f5fa-def2-4b20-9317-750e47b6abbd admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.715097 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f5fa-def2-4b20-9317-750e47b6abbd admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.715373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461f5fa-def2-4b20-9317-750e47b6abbd admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.716125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461f5fa-def2-4b20-9317-750e47b6abbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:19.717723 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461f5fa-def2-4b20-9317-750e47b6abbd admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:19.728053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 980/1956] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:19 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:19.732404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfd2d9-ce30-4b1b-96a4-9bd9d4b440a5 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:19.744142 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfd2d9-ce30-4b1b-96a4-9bd9d4b440a5 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:19.749262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4dbce-a737-47ed-9739-aa12b4799dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:19.749950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4dbce-a737-47ed-9739-aa12b4799dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.750231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4dbce-a737-47ed-9739-aa12b4799dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.750523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4dbce-a737-47ed-9739-aa12b4799dbe admin admin] PATH_INFO: `/v3/projects/82c7f9291b174fa09c4d85e891d986cf/users/83407d7e8c114a9c8290406f49891e38/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.767537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4dbce-a737-47ed-9739-aa12b4799dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c7f9291b174fa09c4d85e891d986cf, user_id=83407d7e8c114a9c8290406f49891e38, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:19.770321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4dbce-a737-47ed-9739-aa12b4799dbe admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:19.871262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 977/1957] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:19 2022] PUT /identity/v3/projects/82c7f9291b174fa09c4d85e891d986cf/users/83407d7e8c114a9c8290406f49891e38/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:19.882619 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70a0a6-7a92-4cdd-85ea-34707d0a5d99 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:19.882928 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70a0a6-7a92-4cdd-85ea-34707d0a5d99 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:19.883260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70a0a6-7a92-4cdd-85ea-34707d0a5d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:19.912452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-6a70a0a6-7a92-4cdd-85ea-34707d0a5d99 None None] MFA Rules not processed for user `83407d7e8c114a9c8290406f49891e38`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:19.913734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6a70a0a6-7a92-4cdd-85ea-34707d0a5d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:19.979375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 981/1958] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:19.990391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4dbce-a737-47ed-9739-aa12b4799dbe admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:19.992104 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9eb4dbce-a737-47ed-9739-aa12b4799dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:20.034300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:20.052367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4dbce-a737-47ed-9739-aa12b4799dbe admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:20.056075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b01efd-c291-447a-8c93-c3fd12d0f940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:20.056676 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b01efd-c291-447a-8c93-c3fd12d0f940 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:20.056955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b01efd-c291-447a-8c93-c3fd12d0f940 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:20.057223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b01efd-c291-447a-8c93-c3fd12d0f940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:20.065863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b01efd-c291-447a-8c93-c3fd12d0f940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:20.070145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b01efd-c291-447a-8c93-c3fd12d0f940 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:20.075936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 978/1959] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:25.274926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70a0a6-7a92-4cdd-85ea-34707d0a5d99 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:25.276398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6a70a0a6-7a92-4cdd-85ea-34707d0a5d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:25.322475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:25.341514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70a0a6-7a92-4cdd-85ea-34707d0a5d99 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:25.346355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:25.346982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:25.347285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:25.347553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] PATH_INFO: `/v3/users/ae357b6df430406e9838c10f12d0311a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:25.348655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae357b6df430406e9838c10f12d0311a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:25.350594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:25.482532 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c071751-0907-424d-880e-abdc24e24b7d None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:25.483113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c071751-0907-424d-880e-abdc24e24b7d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:25.483113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c071751-0907-424d-880e-abdc24e24b7d None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:25.484439 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 979/1960] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 05:01:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 05:01:25.496463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e71c1-9557-45bb-a09d-c001d2ac6d75 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:25.502047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e71c1-9557-45bb-a09d-c001d2ac6d75 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:25.502047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04e71c1-9557-45bb-a09d-c001d2ac6d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:25.518241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae357b6df430406e9838c10f12d0311a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:25.519340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae357b6df430406e9838c10f12d0311a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:25.520469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae357b6df430406e9838c10f12d0311a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:25.539220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c04e71c1-9557-45bb-a09d-c001d2ac6d75 None None] MFA Rules not processed for user `ad8825c294534a7c88877b48b5f8f07f`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:25.540746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c04e71c1-9557-45bb-a09d-c001d2ac6d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:25.607920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 980/1961] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 05:01:25 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:25.624266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e71c1-9557-45bb-a09d-c001d2ac6d75 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:25.624266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c04e71c1-9557-45bb-a09d-c001d2ac6d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:25.634717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04e71c1-9557-45bb-a09d-c001d2ac6d75 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:25.639949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e20a2-01c6-40c9-b4cd-37d155efd284 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:25.641071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e20a2-01c6-40c9-b4cd-37d155efd284 service placement] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:25.641071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e20a2-01c6-40c9-b4cd-37d155efd284 service placement] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:25.641262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e20a2-01c6-40c9-b4cd-37d155efd284 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:25.647858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae357b6df430406e9838c10f12d0311a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:25.650460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e20a2-01c6-40c9-b4cd-37d155efd284 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:25.652986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314e20a2-01c6-40c9-b4cd-37d155efd284 service placement] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:25.662969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae357b6df430406e9838c10f12d0311a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:25.665309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 981/1962] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:25 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:25.681968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 982/1963] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:25 2022] DELETE /identity/v3/users/ae357b6df430406e9838c10f12d0311a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:25.685619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6737e8-79f2-4143-97c3-3a0383fcb64f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:25.685906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6737e8-79f2-4143-97c3-3a0383fcb64f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:25.686179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6737e8-79f2-4143-97c3-3a0383fcb64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:25.705879 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2d6737e8-79f2-4143-97c3-3a0383fcb64f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:25.707106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2d6737e8-79f2-4143-97c3-3a0383fcb64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:25.729495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:25.731118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:25.773772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 982/1964] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:25.785021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6737e8-79f2-4143-97c3-3a0383fcb64f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:25.786528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2d6737e8-79f2-4143-97c3-3a0383fcb64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:25.803939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6137b-af1d-4950-89dd-4e5d0f2e5e2e admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:25.808658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:25.809289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:25.809571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:25.809868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] PATH_INFO: `/v3/users/312b0174466b4b00be19d78bdc01fdb4` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:25.811000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] RBAC: Authorizing `identity:delete_user(user_id=312b0174466b4b00be19d78bdc01fdb4)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:25.813030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:25.844663 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6737e8-79f2-4143-97c3-3a0383fcb64f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:25.848317 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a1e1e-e26c-4898-b0f2-538876826ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:25.848976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a1e1e-e26c-4898-b0f2-538876826ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:25.849223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a1e1e-e26c-4898-b0f2-538876826ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:25.849473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860a1e1e-e26c-4898-b0f2-538876826ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:25.858772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a1e1e-e26c-4898-b0f2-538876826ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:25.860266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860a1e1e-e26c-4898-b0f2-538876826ba8 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:25.869775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 983/1965] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:26.012052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312b0174466b4b00be19d78bdc01fdb4'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.012782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312b0174466b4b00be19d78bdc01fdb4'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.013459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312b0174466b4b00be19d78bdc01fdb4'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.110972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312b0174466b4b00be19d78bdc01fdb4'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.121826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:26.127983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312b0174466b4b00be19d78bdc01fdb4'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.143675 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 983/1966] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:25 2022] DELETE /identity/v3/users/312b0174466b4b00be19d78bdc01fdb4 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:26.334298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a1e1e-e26c-4898-b0f2-538876826ba8 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:26.335734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-860a1e1e-e26c-4898-b0f2-538876826ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:26.344869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:26.349853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860a1e1e-e26c-4898-b0f2-538876826ba8 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:26.411529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:26.412192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:26.412454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:26.412728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] PATH_INFO: `/v3/projects/8285a764591941dfaa26ac5a9d1a5b1b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:26.413853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8285a764591941dfaa26ac5a9d1a5b1b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:26.415809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:26.450729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:26.452469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:26.467686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984806d6-4673-4d44-88a0-2a10f6b7e9da admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:26.474911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:26.474911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:26.474911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:26.474911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] PATH_INFO: `/v3/projects/ae5545057ecc4c02817cd846dc73b55b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:26.475637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae5545057ecc4c02817cd846dc73b55b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:26.477575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:26.590905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8285a764591941dfaa26ac5a9d1a5b1b', 'request_id': 'req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.598088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8285a764591941dfaa26ac5a9d1a5b1b', 'request_id': 'req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.673377 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae5545057ecc4c02817cd846dc73b55b', 'request_id': 'req-500dac2c-01b0-4655-8708-c623b20874d6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.680245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae5545057ecc4c02817cd846dc73b55b', 'request_id': 'req-500dac2c-01b0-4655-8708-c623b20874d6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.686408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae5545057ecc4c02817cd846dc73b55b', 'request_id': 'req-500dac2c-01b0-4655-8708-c623b20874d6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.728710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8285a764591941dfaa26ac5a9d1a5b1b', 'request_id': 'req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.741019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] The token cache is being invalidate because project 8285a764591941dfaa26ac5a9d1a5b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:26.741308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.741945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.743521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 984/1967] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:26 2022] DELETE /identity/v3/projects/8285a764591941dfaa26ac5a9d1a5b1b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:26.747506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:26.748890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:26.818816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfee7f5-9992-4679-a8a8-d3e78a50d2c3 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:26.848360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] The token cache is being invalidate because project ae5545057ecc4c02817cd846dc73b55b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:26.848653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.850001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-500dac2c-01b0-4655-8708-c623b20874d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:26.851967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 984/1968] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:26 2022] DELETE /identity/v3/projects/ae5545057ecc4c02817cd846dc73b55b => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:26.864163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211986f-202f-41a6-a748-36fe869151b5 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:26.864655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211986f-202f-41a6-a748-36fe869151b5 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:26.865091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211986f-202f-41a6-a748-36fe869151b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:26.868728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:26.886803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:26.888541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:26.888541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:26.888541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] PATH_INFO: `/v3/users/af0881cb1e31492dae24947ac7f4c226` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:26.889384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0881cb1e31492dae24947ac7f4c226)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:26.892070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:26.908987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-9211986f-202f-41a6-a748-36fe869151b5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:26.910143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9211986f-202f-41a6-a748-36fe869151b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:26.918311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 985/1969] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:26.928113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78df888-3bf4-4a95-bb3c-3498fefdb58c None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:26.928869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78df888-3bf4-4a95-bb3c-3498fefdb58c None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:26.930268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78df888-3bf4-4a95-bb3c-3498fefdb58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:26.953264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d78df888-3bf4-4a95-bb3c-3498fefdb58c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:26.954895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d78df888-3bf4-4a95-bb3c-3498fefdb58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:26.963410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 986/1970] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:26.973128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78df888-3bf4-4a95-bb3c-3498fefdb58c None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:26.982828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78df888-3bf4-4a95-bb3c-3498fefdb58c None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:26.988387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc2f99-f2f3-47d0-8b7e-80e36284d558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:26.989069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc2f99-f2f3-47d0-8b7e-80e36284d558 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:26.989424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc2f99-f2f3-47d0-8b7e-80e36284d558 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:26.989816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cc2f99-f2f3-47d0-8b7e-80e36284d558 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:26.990539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc2f99-f2f3-47d0-8b7e-80e36284d558 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:01:26.990952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc2f99-f2f3-47d0-8b7e-80e36284d558 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:26.992655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cc2f99-f2f3-47d0-8b7e-80e36284d558 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:27.005540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 987/1971] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:01:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:01:27.009974 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc2f99-f2f3-47d0-8b7e-80e36284d558 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:27.019994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cc2f99-f2f3-47d0-8b7e-80e36284d558 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:27.024501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb0c9-0f8c-4cee-be16-02dd275005f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:27.025079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb0c9-0f8c-4cee-be16-02dd275005f9 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:27.025361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb0c9-0f8c-4cee-be16-02dd275005f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:27.025636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbb0c9-0f8c-4cee-be16-02dd275005f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:27.026418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbb0c9-0f8c-4cee-be16-02dd275005f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:27.028364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbb0c9-0f8c-4cee-be16-02dd275005f9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:27.106884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 988/1972] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:27 2022] POST /identity/v3/projects => generated 374 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:27.111516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb0c9-0f8c-4cee-be16-02dd275005f9 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:27.115250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0881cb1e31492dae24947ac7f4c226'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:27.116074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0881cb1e31492dae24947ac7f4c226'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:27.116683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0881cb1e31492dae24947ac7f4c226'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:27.181235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbb0c9-0f8c-4cee-be16-02dd275005f9 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:27.187860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96d3a9-2f6b-4ad6-aed4-18af24a3bc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:27.189170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d3a9-2f6b-4ad6-aed4-18af24a3bc4f admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:27.189451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d3a9-2f6b-4ad6-aed4-18af24a3bc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:27.189724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb96d3a9-2f6b-4ad6-aed4-18af24a3bc4f admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:27.190355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96d3a9-2f6b-4ad6-aed4-18af24a3bc4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:27.192486 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb96d3a9-2f6b-4ad6-aed4-18af24a3bc4f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:27.219173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0881cb1e31492dae24947ac7f4c226'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:27.232392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0881cb1e31492dae24947ac7f4c226'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:27.246431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 985/1973] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:26 2022] DELETE /identity/v3/users/af0881cb1e31492dae24947ac7f4c226 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:27.276842 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 989/1974] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:27 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:27.281241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:27.283917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:27.362630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efed3a0d-307b-47e6-90b5-1f4de61b6c4f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:27.368582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8a35c-8605-4feb-94f1-f4e660b1a38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:27.369193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8a35c-8605-4feb-94f1-f4e660b1a38b admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:27.369472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8a35c-8605-4feb-94f1-f4e660b1a38b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:27.370098 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8a35c-8605-4feb-94f1-f4e660b1a38b admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:27.370866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8a35c-8605-4feb-94f1-f4e660b1a38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:27.371996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8a35c-8605-4feb-94f1-f4e660b1a38b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:27.382737 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 986/1975] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:27 2022] GET /identity/v3/roles => generated 1427 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:01:27.387171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96d3a9-2f6b-4ad6-aed4-18af24a3bc4f admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:27.397679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb96d3a9-2f6b-4ad6-aed4-18af24a3bc4f admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:27.402461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ca3d46-4d57-4682-9061-8d12f0b6fa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:27.403420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca3d46-4d57-4682-9061-8d12f0b6fa20 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:27.403705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca3d46-4d57-4682-9061-8d12f0b6fa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:27.403980 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca3d46-4d57-4682-9061-8d12f0b6fa20 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:27.404655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca3d46-4d57-4682-9061-8d12f0b6fa20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:27.406383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ca3d46-4d57-4682-9061-8d12f0b6fa20 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:27.408438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=87,keystone:DELETE=8,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:27.417911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 990/1976] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:27 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:27.422268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8a35c-8605-4feb-94f1-f4e660b1a38b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:27.434779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8a35c-8605-4feb-94f1-f4e660b1a38b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:27.440303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e750b5b-994c-48a7-8ff0-2cea4c1e4204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:27.441252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e750b5b-994c-48a7-8ff0-2cea4c1e4204 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:27.441252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e750b5b-994c-48a7-8ff0-2cea4c1e4204 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:27.441539 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e750b5b-994c-48a7-8ff0-2cea4c1e4204 admin admin] PATH_INFO: `/v3/projects/0f8c9d23baa4412cb6fc1e1749cd5212/users/47eff05ff6b64972a00031fd74ebf211/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:27.470358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e750b5b-994c-48a7-8ff0-2cea4c1e4204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8c9d23baa4412cb6fc1e1749cd5212, user_id=47eff05ff6b64972a00031fd74ebf211, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:27.474501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e750b5b-994c-48a7-8ff0-2cea4c1e4204 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:27.617345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 987/1977] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:27 2022] PUT /identity/v3/projects/0f8c9d23baa4412cb6fc1e1749cd5212/users/47eff05ff6b64972a00031fd74ebf211/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:27.619484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245a51f-eefb-4af1-8be4-4fec740e4659 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:27.619925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245a51f-eefb-4af1-8be4-4fec740e4659 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:27.620046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245a51f-eefb-4af1-8be4-4fec740e4659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:27.657513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c245a51f-eefb-4af1-8be4-4fec740e4659 None None] MFA Rules not processed for user `47eff05ff6b64972a00031fd74ebf211`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:27.659192 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c245a51f-eefb-4af1-8be4-4fec740e4659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:27.738596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 991/1978] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:27.762042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e750b5b-994c-48a7-8ff0-2cea4c1e4204 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:27.764058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4e750b5b-994c-48a7-8ff0-2cea4c1e4204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:27.804995 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:27.843999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e750b5b-994c-48a7-8ff0-2cea4c1e4204 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:27.852096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e0087-3290-403e-aa7c-804a25a2f88c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:27.853928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0087-3290-403e-aa7c-804a25a2f88c service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:27.853928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0087-3290-403e-aa7c-804a25a2f88c service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:27.853928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e0087-3290-403e-aa7c-804a25a2f88c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:27.863311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e0087-3290-403e-aa7c-804a25a2f88c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:27.864910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e0087-3290-403e-aa7c-804a25a2f88c service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:27.874407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 988/1979] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:28.021257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245a51f-eefb-4af1-8be4-4fec740e4659 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:28.022933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c245a51f-eefb-4af1-8be4-4fec740e4659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:28.105466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245a51f-eefb-4af1-8be4-4fec740e4659 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:28.112180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:28.112830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.113160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.113441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] PATH_INFO: `/v3/projects/faa822654d0445f796c78b170fbce9ae` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.114590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=faa822654d0445f796c78b170fbce9ae)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:28.116626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:28.148910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e0087-3290-403e-aa7c-804a25a2f88c service nova] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:28.159657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e0087-3290-403e-aa7c-804a25a2f88c service nova] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:28.164719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:28.164846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.165020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.165290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] PATH_INFO: `/v3/users/47eff05ff6b64972a00031fd74ebf211` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.166402 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] RBAC: Authorizing `identity:delete_user(user_id=47eff05ff6b64972a00031fd74ebf211)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:28.168326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:28.231361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faa822654d0445f796c78b170fbce9ae', 'request_id': 'req-c63121ac-5be6-4342-9555-7cb6139c51ab'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:28.237323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faa822654d0445f796c78b170fbce9ae', 'request_id': 'req-c63121ac-5be6-4342-9555-7cb6139c51ab'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:28.242828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faa822654d0445f796c78b170fbce9ae', 'request_id': 'req-c63121ac-5be6-4342-9555-7cb6139c51ab'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:28.350617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] The token cache is being invalidate because project faa822654d0445f796c78b170fbce9ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:28.350910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:28.351548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c63121ac-5be6-4342-9555-7cb6139c51ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:28.353073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 992/1980] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:28 2022] DELETE /identity/v3/projects/faa822654d0445f796c78b170fbce9ae => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:28.358680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f16f43-1b56-4553-b8d3-cd6ef82956e4 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.359038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f16f43-1b56-4553-b8d3-cd6ef82956e4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.359334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f16f43-1b56-4553-b8d3-cd6ef82956e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.379523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-68f16f43-1b56-4553-b8d3-cd6ef82956e4 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:28.382450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-68f16f43-1b56-4553-b8d3-cd6ef82956e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:28.393316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:28.395584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47eff05ff6b64972a00031fd74ebf211'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:28.396240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47eff05ff6b64972a00031fd74ebf211'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:28.396863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47eff05ff6b64972a00031fd74ebf211'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:28.443280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 993/1981] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:28.450643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d13ac-7ffb-4d71-98ec-0149348ec66f None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.450930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d13ac-7ffb-4d71-98ec-0149348ec66f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.451229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d13ac-7ffb-4d71-98ec-0149348ec66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.475791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2d2d13ac-7ffb-4d71-98ec-0149348ec66f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:28.477004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2d2d13ac-7ffb-4d71-98ec-0149348ec66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:28.485432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 994/1982] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:28.489777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d13ac-7ffb-4d71-98ec-0149348ec66f None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:28.498502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d13ac-7ffb-4d71-98ec-0149348ec66f None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:28.503860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244bbe3-1952-4b18-94ed-ccf8c762cfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:28.504447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244bbe3-1952-4b18-94ed-ccf8c762cfe8 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.504725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244bbe3-1952-4b18-94ed-ccf8c762cfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.504996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e244bbe3-1952-4b18-94ed-ccf8c762cfe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.505641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244bbe3-1952-4b18-94ed-ccf8c762cfe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:01:28.505939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244bbe3-1952-4b18-94ed-ccf8c762cfe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:28.507557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e244bbe3-1952-4b18-94ed-ccf8c762cfe8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:28.518853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 995/1983] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:01:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:01:28.523067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244bbe3-1952-4b18-94ed-ccf8c762cfe8 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:28.533329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e244bbe3-1952-4b18-94ed-ccf8c762cfe8 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:28.538862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47eff05ff6b64972a00031fd74ebf211'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:28.539618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c6455-60e2-412b-add4-1ab8c28f598b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:28.540232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c6455-60e2-412b-add4-1ab8c28f598b admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.540508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c6455-60e2-412b-add4-1ab8c28f598b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.540797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c6455-60e2-412b-add4-1ab8c28f598b admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.541416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c6455-60e2-412b-add4-1ab8c28f598b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:28.543462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c6455-60e2-412b-add4-1ab8c28f598b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:28.552241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-95d83db8-0383-4cd2-9c56-721b28d9679e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47eff05ff6b64972a00031fd74ebf211'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:28.564881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 989/1984] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:28 2022] DELETE /identity/v3/users/47eff05ff6b64972a00031fd74ebf211 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:28.569158 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f75f6-0e41-4f42-91de-28321ad92762 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.569446 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f75f6-0e41-4f42-91de-28321ad92762 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.569750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f75f6-0e41-4f42-91de-28321ad92762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.588613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-618f75f6-0e41-4f42-91de-28321ad92762 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:28.589858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-618f75f6-0e41-4f42-91de-28321ad92762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:28.653527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 990/1985] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:28.663505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f75f6-0e41-4f42-91de-28321ad92762 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:28.665018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-618f75f6-0e41-4f42-91de-28321ad92762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:28.693398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 996/1986] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:28 2022] POST /identity/v3/projects => generated 402 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:28.695258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c6455-60e2-412b-add4-1ab8c28f598b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:28.723939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f75f6-0e41-4f42-91de-28321ad92762 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:28.757718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c6455-60e2-412b-add4-1ab8c28f598b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:28.762357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610d28a-6067-406d-8898-7d1793df040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:28.762959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610d28a-6067-406d-8898-7d1793df040d admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.763263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610d28a-6067-406d-8898-7d1793df040d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.763530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610d28a-6067-406d-8898-7d1793df040d admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.764164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610d28a-6067-406d-8898-7d1793df040d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:28.766125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610d28a-6067-406d-8898-7d1793df040d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:28.772922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f690826-9d93-4422-82f8-4b1407ae8303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:28.773605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f690826-9d93-4422-82f8-4b1407ae8303 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.773902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f690826-9d93-4422-82f8-4b1407ae8303 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.774189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f690826-9d93-4422-82f8-4b1407ae8303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.784149 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f690826-9d93-4422-82f8-4b1407ae8303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:28.785645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f690826-9d93-4422-82f8-4b1407ae8303 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:28.795345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 991/1987] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:28.886730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 997/1988] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:28 2022] POST /identity/v3/users => generated 302 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:28.890500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f690826-9d93-4422-82f8-4b1407ae8303 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:28.901029 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f690826-9d93-4422-82f8-4b1407ae8303 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:28.905883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30032f-51c3-4831-9556-f5203689c358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:28.906488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30032f-51c3-4831-9556-f5203689c358 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.907336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30032f-51c3-4831-9556-f5203689c358 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.907824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea30032f-51c3-4831-9556-f5203689c358 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.908467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30032f-51c3-4831-9556-f5203689c358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:28.910096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea30032f-51c3-4831-9556-f5203689c358 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:28.921000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 992/1989] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:28 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:28.926188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610d28a-6067-406d-8898-7d1793df040d admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:28.936606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610d28a-6067-406d-8898-7d1793df040d admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:28.944003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571065cb-635c-4eeb-8c3d-f05351778751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:28.944003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571065cb-635c-4eeb-8c3d-f05351778751 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.944003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571065cb-635c-4eeb-8c3d-f05351778751 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.944003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571065cb-635c-4eeb-8c3d-f05351778751 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.944413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571065cb-635c-4eeb-8c3d-f05351778751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:28.945635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571065cb-635c-4eeb-8c3d-f05351778751 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:28.960396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 998/1990] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:28 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:28.962149 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30032f-51c3-4831-9556-f5203689c358 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:28.974507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea30032f-51c3-4831-9556-f5203689c358 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:28.980400 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498aacf-0c16-4cbb-afdd-5a2502b49186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:28.981665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498aacf-0c16-4cbb-afdd-5a2502b49186 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:28.982357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498aacf-0c16-4cbb-afdd-5a2502b49186 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:28.982494 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498aacf-0c16-4cbb-afdd-5a2502b49186 admin admin] PATH_INFO: `/v3/projects/24a51689c5d1404e8745688ad9ef3902/users/51a2672697024fd4821d1ad385b4daef/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:28.999829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498aacf-0c16-4cbb-afdd-5a2502b49186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a51689c5d1404e8745688ad9ef3902, user_id=51a2672697024fd4821d1ad385b4daef, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:29.003086 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498aacf-0c16-4cbb-afdd-5a2502b49186 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:29.117989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 993/1991] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:28 2022] PUT /identity/v3/projects/24a51689c5d1404e8745688ad9ef3902/users/51a2672697024fd4821d1ad385b4daef/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:29.133696 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a5370-0183-42e9-b2c9-e7b7494fe068 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.134365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a5370-0183-42e9-b2c9-e7b7494fe068 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.134658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882a5370-0183-42e9-b2c9-e7b7494fe068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.136554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:29.168589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-882a5370-0183-42e9-b2c9-e7b7494fe068 None None] MFA Rules not processed for user `51a2672697024fd4821d1ad385b4daef`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:29.169825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-882a5370-0183-42e9-b2c9-e7b7494fe068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:29.251893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 999/1992] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:29 2022] POST /identity/v3/auth/tokens => generated 3288 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:29.255229 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498aacf-0c16-4cbb-afdd-5a2502b49186 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:29.279408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a5370-0183-42e9-b2c9-e7b7494fe068 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:29.281553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-882a5370-0183-42e9-b2c9-e7b7494fe068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:29.330895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498aacf-0c16-4cbb-afdd-5a2502b49186 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:29.336913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc811753-c36e-438f-856c-fc6627b2832c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:29.338352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc811753-c36e-438f-856c-fc6627b2832c admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.338887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc811753-c36e-438f-856c-fc6627b2832c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.339804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc811753-c36e-438f-856c-fc6627b2832c admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.340612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc811753-c36e-438f-856c-fc6627b2832c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:29.346770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc811753-c36e-438f-856c-fc6627b2832c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:29.415796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882a5370-0183-42e9-b2c9-e7b7494fe068 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:29.419263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:29.419961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.420318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.420654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] PATH_INFO: `/v3/projects/0f8c9d23baa4412cb6fc1e1749cd5212` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.421864 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8c9d23baa4412cb6fc1e1749cd5212)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:29.423896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:29.465047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:29.496241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 994/1993] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:29 2022] POST /identity/v3/projects => generated 400 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:29.504332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc811753-c36e-438f-856c-fc6627b2832c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:29.537455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8c9d23baa4412cb6fc1e1749cd5212', 'request_id': 'req-1fb80e6f-7027-4e55-856a-81e4c809432a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:29.545921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8c9d23baa4412cb6fc1e1749cd5212', 'request_id': 'req-1fb80e6f-7027-4e55-856a-81e4c809432a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:29.552863 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8c9d23baa4412cb6fc1e1749cd5212', 'request_id': 'req-1fb80e6f-7027-4e55-856a-81e4c809432a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:29.577679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc811753-c36e-438f-856c-fc6627b2832c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:29.631087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] The token cache is being invalidate because project 0f8c9d23baa4412cb6fc1e1749cd5212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:29.631391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:29.632121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1fb80e6f-7027-4e55-856a-81e4c809432a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:29.633657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1000/1994] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:29 2022] DELETE /identity/v3/projects/0f8c9d23baa4412cb6fc1e1749cd5212 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:29.636152 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0765c-1307-4955-80bb-a742c5e82000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:29.636823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0765c-1307-4955-80bb-a742c5e82000 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.637177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0765c-1307-4955-80bb-a742c5e82000 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.637609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0765c-1307-4955-80bb-a742c5e82000 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.638409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0765c-1307-4955-80bb-a742c5e82000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:29.638677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791530e-5b45-4a9c-8061-c6bde8683362 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.638968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791530e-5b45-4a9c-8061-c6bde8683362 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.639339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1791530e-5b45-4a9c-8061-c6bde8683362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.641025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0765c-1307-4955-80bb-a742c5e82000 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:29.660228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-1791530e-5b45-4a9c-8061-c6bde8683362 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:29.661451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1791530e-5b45-4a9c-8061-c6bde8683362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:29.669689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1001/1995] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:29.679134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fa22e-5605-4511-a11a-d52b8a00bb61 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.679421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fa22e-5605-4511-a11a-d52b8a00bb61 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.679692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fa22e-5605-4511-a11a-d52b8a00bb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.701846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-893fa22e-5605-4511-a11a-d52b8a00bb61 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:29.703183 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-893fa22e-5605-4511-a11a-d52b8a00bb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:29.711721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1002/1996] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:29.716020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fa22e-5605-4511-a11a-d52b8a00bb61 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:29.725167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fa22e-5605-4511-a11a-d52b8a00bb61 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:29.729921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529755a-4e3c-401d-911c-f254136b2114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:29.730513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529755a-4e3c-401d-911c-f254136b2114 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.730796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529755a-4e3c-401d-911c-f254136b2114 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.731111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529755a-4e3c-401d-911c-f254136b2114 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.731848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529755a-4e3c-401d-911c-f254136b2114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:01:29.732122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529755a-4e3c-401d-911c-f254136b2114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:29.735445 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529755a-4e3c-401d-911c-f254136b2114 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:29.740463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 995/1997] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:29 2022] POST /identity/v3/users => generated 307 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:29.748091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0765c-1307-4955-80bb-a742c5e82000 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:29.750034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-fea0765c-1307-4955-80bb-a742c5e82000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:29.751136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1003/1998] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:01:29 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:01:29.769931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529755a-4e3c-401d-911c-f254136b2114 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:29.770157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0765c-1307-4955-80bb-a742c5e82000 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:29.770248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529755a-4e3c-401d-911c-f254136b2114 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:29.774287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d9c91-bff4-4369-a4ed-189e53bad592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:29.774912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d9c91-bff4-4369-a4ed-189e53bad592 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.775230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d9c91-bff4-4369-a4ed-189e53bad592 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.775503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d9c91-bff4-4369-a4ed-189e53bad592 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.775843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18715ef-3466-4ac6-bc84-57067abc1324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:29.776242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d9c91-bff4-4369-a4ed-189e53bad592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:29.776931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18715ef-3466-4ac6-bc84-57067abc1324 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.777368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18715ef-3466-4ac6-bc84-57067abc1324 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.777786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18715ef-3466-4ac6-bc84-57067abc1324 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.778053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d9c91-bff4-4369-a4ed-189e53bad592 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:29.778719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18715ef-3466-4ac6-bc84-57067abc1324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:29.780808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18715ef-3466-4ac6-bc84-57067abc1324 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:29.792866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 996/1999] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:29 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:01:29.793697 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d9c91-bff4-4369-a4ed-189e53bad592 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:29.803981 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d9c91-bff4-4369-a4ed-189e53bad592 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:29.808850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffeaaad-04ca-40e5-9b19-7dfc8d06a38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:29.809609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffeaaad-04ca-40e5-9b19-7dfc8d06a38f admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.809890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffeaaad-04ca-40e5-9b19-7dfc8d06a38f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.810166 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffeaaad-04ca-40e5-9b19-7dfc8d06a38f admin admin] PATH_INFO: `/v3/projects/76758dbeb43241a98659aef117c529ed/users/6d578e457a6d4576a6c3a2d72a5ce197/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.815363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:29.826647 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffeaaad-04ca-40e5-9b19-7dfc8d06a38f admin admin] RBAC: Authorizing `identity:create_grant(project_id=76758dbeb43241a98659aef117c529ed, user_id=6d578e457a6d4576a6c3a2d72a5ce197, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:29.830009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffeaaad-04ca-40e5-9b19-7dfc8d06a38f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:29.872799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1004/2000] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:29 2022] POST /identity/v3/projects => generated 366 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:29.876169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18715ef-3466-4ac6-bc84-57067abc1324 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:29.900989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 997/2001] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:29 2022] PUT /identity/v3/projects/76758dbeb43241a98659aef117c529ed/users/6d578e457a6d4576a6c3a2d72a5ce197/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:29.905192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffeaaad-04ca-40e5-9b19-7dfc8d06a38f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:29.943273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18715ef-3466-4ac6-bc84-57067abc1324 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:29.971172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:29.978099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffeaaad-04ca-40e5-9b19-7dfc8d06a38f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:29.983347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ba689-f5e1-4fb1-b711-7f8416870f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:29.983963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ba689-f5e1-4fb1-b711-7f8416870f38 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:29.984238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ba689-f5e1-4fb1-b711-7f8416870f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:29.984530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1ba689-f5e1-4fb1-b711-7f8416870f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:29.985208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ba689-f5e1-4fb1-b711-7f8416870f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:29.986809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1ba689-f5e1-4fb1-b711-7f8416870f38 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:29.999128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 998/2002] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:29 2022] GET /identity/v3/roles => generated 1427 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:01:30.003218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ba689-f5e1-4fb1-b711-7f8416870f38 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:30.005208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129924c6-41e2-43ce-8467-11ca84677ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:30.005827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129924c6-41e2-43ce-8467-11ca84677ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.006155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129924c6-41e2-43ce-8467-11ca84677ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.006390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129924c6-41e2-43ce-8467-11ca84677ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.006974 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129924c6-41e2-43ce-8467-11ca84677ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:30.014564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129924c6-41e2-43ce-8467-11ca84677ae2 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:30.014664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1ba689-f5e1-4fb1-b711-7f8416870f38 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:30.019367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa930cc5-22bf-47a4-9947-cb677599fc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:30.019738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa930cc5-22bf-47a4-9947-cb677599fc94 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.020023 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa930cc5-22bf-47a4-9947-cb677599fc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.020302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa930cc5-22bf-47a4-9947-cb677599fc94 admin admin] PATH_INFO: `/v3/projects/76758dbeb43241a98659aef117c529ed/users/6d578e457a6d4576a6c3a2d72a5ce197/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.022252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa930cc5-22bf-47a4-9947-cb677599fc94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76758dbeb43241a98659aef117c529ed, user_id=6d578e457a6d4576a6c3a2d72a5ce197, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:30.025004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa930cc5-22bf-47a4-9947-cb677599fc94 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:30.037241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 999/2003] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:30 2022] PUT /identity/v3/projects/76758dbeb43241a98659aef117c529ed/users/6d578e457a6d4576a6c3a2d72a5ce197/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:30.047692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3c75b-0a1b-4a2e-9e77-33718e720dc2 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.048065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3c75b-0a1b-4a2e-9e77-33718e720dc2 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.048421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e3c75b-0a1b-4a2e-9e77-33718e720dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.088609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-07e3c75b-0a1b-4a2e-9e77-33718e720dc2 None None] MFA Rules not processed for user `6d578e457a6d4576a6c3a2d72a5ce197`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:30.089921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-07e3c75b-0a1b-4a2e-9e77-33718e720dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:30.098155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1005/2004] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:29 2022] POST /identity/v3/users => generated 284 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:30.099626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9689aca-ff85-42dc-a1b9-ed25cec1b552 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.099915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9689aca-ff85-42dc-a1b9-ed25cec1b552 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.100190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9689aca-ff85-42dc-a1b9-ed25cec1b552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.124857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f9689aca-ff85-42dc-a1b9-ed25cec1b552 None None] MFA Rules not processed for user `946a7533230b446f8ff34a0a51d4c2f2`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:30.126717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f9689aca-ff85-42dc-a1b9-ed25cec1b552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:30.169992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1000/2005] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:30 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:30.175132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3c75b-0a1b-4a2e-9e77-33718e720dc2 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:30.204113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1006/2006] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:30.204320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:30.204840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9689aca-ff85-42dc-a1b9-ed25cec1b552 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:30.206735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f9689aca-ff85-42dc-a1b9-ed25cec1b552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:30.266799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e3c75b-0a1b-4a2e-9e77-33718e720dc2 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:30.272364 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcfec9-29b5-4853-a005-4c7ce51d497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:30.273050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcfec9-29b5-4853-a005-4c7ce51d497e admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.273389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcfec9-29b5-4853-a005-4c7ce51d497e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.273724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fcfec9-29b5-4853-a005-4c7ce51d497e admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.274431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcfec9-29b5-4853-a005-4c7ce51d497e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:30.276181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fcfec9-29b5-4853-a005-4c7ce51d497e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:30.290879 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1001/2007] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:30 2022] GET /identity/v3/roles => generated 1427 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:30.301658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9689aca-ff85-42dc-a1b9-ed25cec1b552 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:30.304636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcfec9-29b5-4853-a005-4c7ce51d497e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:30.309862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb1994-bd2c-439b-8b44-c6373693ce9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:30.310916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb1994-bd2c-439b-8b44-c6373693ce9e service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.311358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb1994-bd2c-439b-8b44-c6373693ce9e service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.311729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb1994-bd2c-439b-8b44-c6373693ce9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.318893 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fcfec9-29b5-4853-a005-4c7ce51d497e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:30.324816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb1994-bd2c-439b-8b44-c6373693ce9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:30.326504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb1994-bd2c-439b-8b44-c6373693ce9e service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:30.327173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a088e0-df83-4b2e-8a1a-548b17612037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:30.327939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a088e0-df83-4b2e-8a1a-548b17612037 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.328372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a088e0-df83-4b2e-8a1a-548b17612037 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.329288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a088e0-df83-4b2e-8a1a-548b17612037 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.330311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a088e0-df83-4b2e-8a1a-548b17612037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:30.333185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a088e0-df83-4b2e-8a1a-548b17612037 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:30.341426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1007/2008] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:30 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:30.345262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1002/2009] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:30 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:30.349137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb1994-bd2c-439b-8b44-c6373693ce9e service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:30.366161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb1994-bd2c-439b-8b44-c6373693ce9e service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:30.372759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2e1f8-567d-485f-85b0-06112aaf6738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:30.373859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2e1f8-567d-485f-85b0-06112aaf6738 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.374513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2e1f8-567d-485f-85b0-06112aaf6738 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.375139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2e1f8-567d-485f-85b0-06112aaf6738 admin admin] PATH_INFO: `/v3/projects/0b93eb80e63949d083794f0eab347281/users/e02861592a1149dda955352a8dfff92f/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.379613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:30.394351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2e1f8-567d-485f-85b0-06112aaf6738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b93eb80e63949d083794f0eab347281, user_id=e02861592a1149dda955352a8dfff92f, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:30.397707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2e1f8-567d-485f-85b0-06112aaf6738 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:30.469643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1008/2010] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:30 2022] PUT /identity/v3/projects/0b93eb80e63949d083794f0eab347281/users/e02861592a1149dda955352a8dfff92f/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:30.479946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50114b-3c6e-4023-9fce-20c00a05200c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.480231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50114b-3c6e-4023-9fce-20c00a05200c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.480557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50114b-3c6e-4023-9fce-20c00a05200c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.512333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7a50114b-3c6e-4023-9fce-20c00a05200c None None] MFA Rules not processed for user `e02861592a1149dda955352a8dfff92f`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:30.513672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7a50114b-3c6e-4023-9fce-20c00a05200c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:30.578246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2e1f8-567d-485f-85b0-06112aaf6738 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:30.580937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9fb2e1f8-567d-485f-85b0-06112aaf6738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:30.644762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1003/2011] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:30.649703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50114b-3c6e-4023-9fce-20c00a05200c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:30.673155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2e1f8-567d-485f-85b0-06112aaf6738 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:30.691005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba14d6-f916-450d-a14b-8a5a5f6ff7a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:30.693082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba14d6-f916-450d-a14b-8a5a5f6ff7a8 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.693370 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba14d6-f916-450d-a14b-8a5a5f6ff7a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.693650 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ba14d6-f916-450d-a14b-8a5a5f6ff7a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.700934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-07ba14d6-f916-450d-a14b-8a5a5f6ff7a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:30.701905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:30.783059 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50114b-3c6e-4023-9fce-20c00a05200c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:30.790197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aa104-41ac-48a4-947a-08c16fd7ecc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:30.795323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aa104-41ac-48a4-947a-08c16fd7ecc7 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:30.795323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aa104-41ac-48a4-947a-08c16fd7ecc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:30.795323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03aa104-41ac-48a4-947a-08c16fd7ecc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:30.795323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03aa104-41ac-48a4-947a-08c16fd7ecc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:30.795323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03aa104-41ac-48a4-947a-08c16fd7ecc7 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:30.798164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba14d6-f916-450d-a14b-8a5a5f6ff7a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:30.800803 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ba14d6-f916-450d-a14b-8a5a5f6ff7a8 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:30.821764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1009/2012] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:30 2022] GET /identity/v3/auth/tokens => generated 3288 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:30.910059 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1004/2013] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:30 2022] POST /identity/v3/projects => generated 364 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:30.914107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba14d6-f916-450d-a14b-8a5a5f6ff7a8 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:30.930327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ba14d6-f916-450d-a14b-8a5a5f6ff7a8 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:31.018084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb249502-8b8a-48a6-84b1-9f89d9c1e24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:31.018765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb249502-8b8a-48a6-84b1-9f89d9c1e24b admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:31.019243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb249502-8b8a-48a6-84b1-9f89d9c1e24b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:31.020234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb249502-8b8a-48a6-84b1-9f89d9c1e24b admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:31.020873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb249502-8b8a-48a6-84b1-9f89d9c1e24b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:31.023365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb249502-8b8a-48a6-84b1-9f89d9c1e24b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:31.129446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1010/2014] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:30 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:31.131988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aa104-41ac-48a4-947a-08c16fd7ecc7 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:31.151712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03aa104-41ac-48a4-947a-08c16fd7ecc7 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:31.161667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b4f71-56ec-4b71-9ccc-b83e346de64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:31.163080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b4f71-56ec-4b71-9ccc-b83e346de64f admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:31.163652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b4f71-56ec-4b71-9ccc-b83e346de64f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:31.164238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b4f71-56ec-4b71-9ccc-b83e346de64f admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:31.165497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b4f71-56ec-4b71-9ccc-b83e346de64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:31.168717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b4f71-56ec-4b71-9ccc-b83e346de64f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:31.197687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1005/2015] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:31 2022] GET /identity/v3/roles => generated 1427 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:31.201554 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb249502-8b8a-48a6-84b1-9f89d9c1e24b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:31.213953 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb249502-8b8a-48a6-84b1-9f89d9c1e24b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:31.220026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c0f7c-c746-4a74-8190-05cbbcd34f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:31.220878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c0f7c-c746-4a74-8190-05cbbcd34f19 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:31.221313 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c0f7c-c746-4a74-8190-05cbbcd34f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:31.221722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c0f7c-c746-4a74-8190-05cbbcd34f19 admin admin] PATH_INFO: `/v3/projects/061db6580cb44860b3557ff495e45ee5/users/b7ba16af8c9043689490166800ea5de1/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:31.244016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c0f7c-c746-4a74-8190-05cbbcd34f19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061db6580cb44860b3557ff495e45ee5, user_id=b7ba16af8c9043689490166800ea5de1, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:31.244792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c0f7c-c746-4a74-8190-05cbbcd34f19 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:31.339665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1011/2016] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:31 2022] PUT /identity/v3/projects/061db6580cb44860b3557ff495e45ee5/users/b7ba16af8c9043689490166800ea5de1/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:31.349421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b4f71-56ec-4b71-9ccc-b83e346de64f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:31.362156 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b4f71-56ec-4b71-9ccc-b83e346de64f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:31.428274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152aad0b-fd69-40b7-91b8-310ed9fd887d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:31.429042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152aad0b-fd69-40b7-91b8-310ed9fd887d admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:31.429346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152aad0b-fd69-40b7-91b8-310ed9fd887d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:31.429612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152aad0b-fd69-40b7-91b8-310ed9fd887d admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:31.430237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152aad0b-fd69-40b7-91b8-310ed9fd887d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:31.431840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152aad0b-fd69-40b7-91b8-310ed9fd887d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:31.443634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1006/2017] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:31 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:31.451296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c0f7c-c746-4a74-8190-05cbbcd34f19 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:31.458848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:31.462897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c0f7c-c746-4a74-8190-05cbbcd34f19 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:31.467775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01459d-5886-4ade-83c5-df850d37b560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:31.468447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01459d-5886-4ade-83c5-df850d37b560 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:31.468960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01459d-5886-4ade-83c5-df850d37b560 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:31.469742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01459d-5886-4ade-83c5-df850d37b560 admin admin] PATH_INFO: `/v3/projects/061db6580cb44860b3557ff495e45ee5/users/b7ba16af8c9043689490166800ea5de1/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:31.472033 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b01459d-5886-4ade-83c5-df850d37b560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=061db6580cb44860b3557ff495e45ee5, user_id=b7ba16af8c9043689490166800ea5de1, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:31.474671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b01459d-5886-4ade-83c5-df850d37b560 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:31.486818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1012/2018] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:31 2022] PUT /identity/v3/projects/061db6580cb44860b3557ff495e45ee5/users/b7ba16af8c9043689490166800ea5de1/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:31.497301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088280b-45ef-44f9-aa6c-7d61722462eb None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:31.497631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088280b-45ef-44f9-aa6c-7d61722462eb None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:31.497919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088280b-45ef-44f9-aa6c-7d61722462eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:31.530760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-f088280b-45ef-44f9-aa6c-7d61722462eb None None] MFA Rules not processed for user `b7ba16af8c9043689490166800ea5de1`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:31.532837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f088280b-45ef-44f9-aa6c-7d61722462eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:31.618091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1007/2019] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:31.675591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01459d-5886-4ade-83c5-df850d37b560 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:31.736401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01459d-5886-4ade-83c5-df850d37b560 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:31.741510 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d364bee-d09d-41de-b2ac-e1453523a5df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:31.742129 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d364bee-d09d-41de-b2ac-e1453523a5df service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:31.742409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d364bee-d09d-41de-b2ac-e1453523a5df service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:31.742706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d364bee-d09d-41de-b2ac-e1453523a5df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:31.753412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d364bee-d09d-41de-b2ac-e1453523a5df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:31.755140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d364bee-d09d-41de-b2ac-e1453523a5df service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:31.765079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1013/2020] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:32.773475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088280b-45ef-44f9-aa6c-7d61722462eb None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:32.795209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:32.836965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088280b-45ef-44f9-aa6c-7d61722462eb None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:32.842730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:32.843493 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:32.843768 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:32.844738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] PATH_INFO: `/v3/users/e02861592a1149dda955352a8dfff92f` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:32.846311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e02861592a1149dda955352a8dfff92f)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:32.848291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:33.124645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e02861592a1149dda955352a8dfff92f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:33.125890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e02861592a1149dda955352a8dfff92f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:33.126550 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e02861592a1149dda955352a8dfff92f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:33.185044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e02861592a1149dda955352a8dfff92f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:33.206843 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e02861592a1149dda955352a8dfff92f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:33.220659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1008/2021] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:32 2022] DELETE /identity/v3/users/e02861592a1149dda955352a8dfff92f => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:33.225193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf9e0e-6626-4113-a317-041976521e32 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:33.225685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf9e0e-6626-4113-a317-041976521e32 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:33.225779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf9e0e-6626-4113-a317-041976521e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:33.245379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d0cf9e0e-6626-4113-a317-041976521e32 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:33.246652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d0cf9e0e-6626-4113-a317-041976521e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:33.315326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1014/2022] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:33.324683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:33.326074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:33.389046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0961d-82ea-47e3-adba-f4548b0b143d admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:33.393509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fab983-fdf1-49e1-aa03-58ba0b0799c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:33.394127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fab983-fdf1-49e1-aa03-58ba0b0799c4 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:33.394404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fab983-fdf1-49e1-aa03-58ba0b0799c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:33.394673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fab983-fdf1-49e1-aa03-58ba0b0799c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:33.404726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fab983-fdf1-49e1-aa03-58ba0b0799c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:33.406663 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fab983-fdf1-49e1-aa03-58ba0b0799c4 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:33.418306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1009/2023] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:33.811840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf9e0e-6626-4113-a317-041976521e32 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:33.813366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d0cf9e0e-6626-4113-a317-041976521e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:33.825476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf9e0e-6626-4113-a317-041976521e32 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:33.842075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:33.842732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:33.843100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:33.843377 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] PATH_INFO: `/v3/projects/0b93eb80e63949d083794f0eab347281` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:33.845079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b93eb80e63949d083794f0eab347281)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:33.846968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:33.957736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b93eb80e63949d083794f0eab347281', 'request_id': 'req-4091d39f-6a8a-4b06-abe1-52996ea931c8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:33.977509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b93eb80e63949d083794f0eab347281', 'request_id': 'req-4091d39f-6a8a-4b06-abe1-52996ea931c8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:33.997444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b93eb80e63949d083794f0eab347281', 'request_id': 'req-4091d39f-6a8a-4b06-abe1-52996ea931c8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:34.097234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] The token cache is being invalidate because project 0b93eb80e63949d083794f0eab347281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:34.097571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:34.099055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:34.105922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fab983-fdf1-49e1-aa03-58ba0b0799c4 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:34.122335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fab983-fdf1-49e1-aa03-58ba0b0799c4 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:34.122441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1015/2024] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:33 2022] DELETE /identity/v3/projects/0b93eb80e63949d083794f0eab347281 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:34.172600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:34.192520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:34.193449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:34.193884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:34.194955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] PATH_INFO: `/v3/users/b7ba16af8c9043689490166800ea5de1` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:34.196157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7ba16af8c9043689490166800ea5de1)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:34.198196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:34.440195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7ba16af8c9043689490166800ea5de1'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:34.440941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7ba16af8c9043689490166800ea5de1'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:34.441698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7ba16af8c9043689490166800ea5de1'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:34.563060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7ba16af8c9043689490166800ea5de1'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:34.588570 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-05aab7c9-3d9f-471e-a9a3-2bcd4ea6cfe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7ba16af8c9043689490166800ea5de1'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:34.602982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1010/2025] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:34 2022] DELETE /identity/v3/users/b7ba16af8c9043689490166800ea5de1 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:34.997592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:34.999144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:35.018770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091d39f-6a8a-4b06-abe1-52996ea931c8 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:35.033054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:35.087399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:35.088153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:35.088437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:35.088794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] PATH_INFO: `/v3/projects/061db6580cb44860b3557ff495e45ee5` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:35.091749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] RBAC: Authorizing `identity:delete_project(project_id=061db6580cb44860b3557ff495e45ee5)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:35.091848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:35.210094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '061db6580cb44860b3557ff495e45ee5', 'request_id': 'req-234a4d67-1557-4f9c-ae23-58309057922d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:35.216694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '061db6580cb44860b3557ff495e45ee5', 'request_id': 'req-234a4d67-1557-4f9c-ae23-58309057922d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:35.223245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '061db6580cb44860b3557ff495e45ee5', 'request_id': 'req-234a4d67-1557-4f9c-ae23-58309057922d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:35.344993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] The token cache is being invalidate because project 061db6580cb44860b3557ff495e45ee5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:35.345197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:35.345876 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-234a4d67-1557-4f9c-ae23-58309057922d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:35.347434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1016/2026] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:34 2022] DELETE /identity/v3/projects/061db6580cb44860b3557ff495e45ee5 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:35.352519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb20ea-6c5b-4e97-b2c4-9c73d8ffbc08 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:35.353030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb20ea-6c5b-4e97-b2c4-9c73d8ffbc08 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:35.353146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb20ea-6c5b-4e97-b2c4-9c73d8ffbc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:35.377287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-3feb20ea-6c5b-4e97-b2c4-9c73d8ffbc08 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:35.377287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3feb20ea-6c5b-4e97-b2c4-9c73d8ffbc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:35.446188 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1011/2027] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:35.455491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3c8f-dfb1-4de5-8239-d93307e0944a None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:35.455881 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3c8f-dfb1-4de5-8239-d93307e0944a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:35.456240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3c8f-dfb1-4de5-8239-d93307e0944a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:35.486741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-9a6a3c8f-dfb1-4de5-8239-d93307e0944a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:35.488052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9a6a3c8f-dfb1-4de5-8239-d93307e0944a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:35.501774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1017/2028] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:35.516405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb20ea-6c5b-4e97-b2c4-9c73d8ffbc08 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:35.518946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb20ea-6c5b-4e97-b2c4-9c73d8ffbc08 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:35.524847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b5384-9a14-48f0-bcd9-ef738c29dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:35.525154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b5384-9a14-48f0-bcd9-ef738c29dee3 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:35.525503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b5384-9a14-48f0-bcd9-ef738c29dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:35.525963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b5384-9a14-48f0-bcd9-ef738c29dee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:35.526578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b5384-9a14-48f0-bcd9-ef738c29dee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:01:35.526998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b5384-9a14-48f0-bcd9-ef738c29dee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:35.528616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b5384-9a14-48f0-bcd9-ef738c29dee3 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:35.541613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1012/2029] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:01:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:01:35.546906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a3c8f-dfb1-4de5-8239-d93307e0944a None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:35.559790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a3c8f-dfb1-4de5-8239-d93307e0944a None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:35.566310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fc05f-c74c-4929-92d8-37480542a66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:35.567045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fc05f-c74c-4929-92d8-37480542a66b admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:35.567384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fc05f-c74c-4929-92d8-37480542a66b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:35.567719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392fc05f-c74c-4929-92d8-37480542a66b admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:35.568412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392fc05f-c74c-4929-92d8-37480542a66b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:35.571175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392fc05f-c74c-4929-92d8-37480542a66b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:35.681432 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1018/2030] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:35 2022] POST /identity/v3/projects => generated 358 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:35.689503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b5384-9a14-48f0-bcd9-ef738c29dee3 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:35.712967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b5384-9a14-48f0-bcd9-ef738c29dee3 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:35.767971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:35.783522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405e363-3ebc-4169-ae5d-ceb7efd332b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:35.788604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405e363-3ebc-4169-ae5d-ceb7efd332b4 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:35.788604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405e363-3ebc-4169-ae5d-ceb7efd332b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:35.788604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405e363-3ebc-4169-ae5d-ceb7efd332b4 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:35.788604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405e363-3ebc-4169-ae5d-ceb7efd332b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:35.791336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405e363-3ebc-4169-ae5d-ceb7efd332b4 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:35.899428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1013/2031] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:35 2022] POST /identity/v3/users => generated 280 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:35.903653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fc05f-c74c-4929-92d8-37480542a66b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:35.915888 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392fc05f-c74c-4929-92d8-37480542a66b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:35.920666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4db1-6805-4f35-bebc-d72a1a8600aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:35.921388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4db1-6805-4f35-bebc-d72a1a8600aa admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:35.921735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4db1-6805-4f35-bebc-d72a1a8600aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:35.922167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4db1-6805-4f35-bebc-d72a1a8600aa admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:35.923247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d4db1-6805-4f35-bebc-d72a1a8600aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:35.925260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d4db1-6805-4f35-bebc-d72a1a8600aa admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:35.939068 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1019/2032] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:35 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:35.942757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405e363-3ebc-4169-ae5d-ceb7efd332b4 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:35.957699 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405e363-3ebc-4169-ae5d-ceb7efd332b4 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:35.963654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fecd3f-d0ce-4a3f-a930-d041c7c311ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:35.964380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fecd3f-d0ce-4a3f-a930-d041c7c311ae admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:35.964867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fecd3f-d0ce-4a3f-a930-d041c7c311ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:35.965187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fecd3f-d0ce-4a3f-a930-d041c7c311ae admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:35.965924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fecd3f-d0ce-4a3f-a930-d041c7c311ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:35.967871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fecd3f-d0ce-4a3f-a930-d041c7c311ae admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:35.980453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1014/2033] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:35 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:35.984445 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4db1-6805-4f35-bebc-d72a1a8600aa admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:35.994999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4db1-6805-4f35-bebc-d72a1a8600aa admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:35.999758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a6206-2984-4ce7-90cc-f390e99f4828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:36.000621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6206-2984-4ce7-90cc-f390e99f4828 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:36.000930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6206-2984-4ce7-90cc-f390e99f4828 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:36.001197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6206-2984-4ce7-90cc-f390e99f4828 admin admin] PATH_INFO: `/v3/projects/0139022fbb4b410695b52da3b45fabc8/users/004fd0d9df6540ffa7054863a5146562/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:36.016953 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a6206-2984-4ce7-90cc-f390e99f4828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0139022fbb4b410695b52da3b45fabc8, user_id=004fd0d9df6540ffa7054863a5146562, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:36.019769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a6206-2984-4ce7-90cc-f390e99f4828 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:36.088861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1020/2034] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:35 2022] PUT /identity/v3/projects/0139022fbb4b410695b52da3b45fabc8/users/004fd0d9df6540ffa7054863a5146562/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:36.100814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c27b6-387f-4dc9-ad61-bd704f22140b None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:36.101051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c27b6-387f-4dc9-ad61-bd704f22140b None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:36.101325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c27b6-387f-4dc9-ad61-bd704f22140b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:36.138277 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-825c27b6-387f-4dc9-ad61-bd704f22140b None None] MFA Rules not processed for user `004fd0d9df6540ffa7054863a5146562`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:36.139655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-825c27b6-387f-4dc9-ad61-bd704f22140b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:36.228779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1015/2035] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:36.233514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a6206-2984-4ce7-90cc-f390e99f4828 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:36.283898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:36.300682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a6206-2984-4ce7-90cc-f390e99f4828 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:36.305853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa31aa-c11c-462f-b602-c9717e7f48c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:36.306581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa31aa-c11c-462f-b602-c9717e7f48c3 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:36.306860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa31aa-c11c-462f-b602-c9717e7f48c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:36.307303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa31aa-c11c-462f-b602-c9717e7f48c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:36.307934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa31aa-c11c-462f-b602-c9717e7f48c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:36.309916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa31aa-c11c-462f-b602-c9717e7f48c3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:36.423982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1021/2036] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:36 2022] POST /identity/v3/projects => generated 358 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:36.425222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c27b6-387f-4dc9-ad61-bd704f22140b None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:36.436410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c27b6-387f-4dc9-ad61-bd704f22140b None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:36.497191 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317594f-446a-44a1-9e7c-62cd0a1153ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:36.497810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317594f-446a-44a1-9e7c-62cd0a1153ea admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:36.498086 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317594f-446a-44a1-9e7c-62cd0a1153ea admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:36.498356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317594f-446a-44a1-9e7c-62cd0a1153ea admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:36.499161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317594f-446a-44a1-9e7c-62cd0a1153ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:36.501685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4317594f-446a-44a1-9e7c-62cd0a1153ea admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:36.608982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1016/2037] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:36 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:36.612938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa31aa-c11c-462f-b602-c9717e7f48c3 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:36.623912 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa31aa-c11c-462f-b602-c9717e7f48c3 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:36.629762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89610076-22ff-493f-a270-3606852ff6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:36.629889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89610076-22ff-493f-a270-3606852ff6fc admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:36.630135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89610076-22ff-493f-a270-3606852ff6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:36.630543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89610076-22ff-493f-a270-3606852ff6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:36.631255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89610076-22ff-493f-a270-3606852ff6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:36.632853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89610076-22ff-493f-a270-3606852ff6fc admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:36.644323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1022/2038] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:36 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:36.649037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317594f-446a-44a1-9e7c-62cd0a1153ea admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:36.658994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4317594f-446a-44a1-9e7c-62cd0a1153ea admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:36.664038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dac2a9-a04b-437e-9f23-c2651c6d6ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:36.664705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dac2a9-a04b-437e-9f23-c2651c6d6ddf admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:36.666406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dac2a9-a04b-437e-9f23-c2651c6d6ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:36.666406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dac2a9-a04b-437e-9f23-c2651c6d6ddf admin admin] PATH_INFO: `/v3/projects/5091062794a94b248c26bbf81d987e30/users/89b2435ed2ca4d0b817af5a8447af389/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:36.681867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dac2a9-a04b-437e-9f23-c2651c6d6ddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5091062794a94b248c26bbf81d987e30, user_id=89b2435ed2ca4d0b817af5a8447af389, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:36.684579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dac2a9-a04b-437e-9f23-c2651c6d6ddf admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:36.775084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89610076-22ff-493f-a270-3606852ff6fc admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:36.776862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-89610076-22ff-493f-a270-3606852ff6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:36.786271 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1017/2039] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:36 2022] PUT /identity/v3/projects/5091062794a94b248c26bbf81d987e30/users/89b2435ed2ca4d0b817af5a8447af389/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:36.790604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89610076-22ff-493f-a270-3606852ff6fc admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:36.792580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dac2a9-a04b-437e-9f23-c2651c6d6ddf admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:36.817726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:36.848746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:36.849377 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:36.849687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:36.849997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] PATH_INFO: `/v3/users/83407d7e8c114a9c8290406f49891e38` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:36.851200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83407d7e8c114a9c8290406f49891e38)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:36.853120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:36.853339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dac2a9-a04b-437e-9f23-c2651c6d6ddf admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:36.858061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6861fe7-8f8e-4752-85d1-b72b54216382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:36.858659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6861fe7-8f8e-4752-85d1-b72b54216382 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:36.858931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6861fe7-8f8e-4752-85d1-b72b54216382 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:36.859231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6861fe7-8f8e-4752-85d1-b72b54216382 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:36.860692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6861fe7-8f8e-4752-85d1-b72b54216382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:36.862384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6861fe7-8f8e-4752-85d1-b72b54216382 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:36.872647 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1018/2040] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:36 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:36.877571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6861fe7-8f8e-4752-85d1-b72b54216382 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:36.888243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6861fe7-8f8e-4752-85d1-b72b54216382 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:36.893774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2241ff1-456a-41c5-ab23-8c27caae110d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:36.894442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2241ff1-456a-41c5-ab23-8c27caae110d admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:36.894750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2241ff1-456a-41c5-ab23-8c27caae110d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:36.895106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2241ff1-456a-41c5-ab23-8c27caae110d admin admin] PATH_INFO: `/v3/projects/5091062794a94b248c26bbf81d987e30/users/89b2435ed2ca4d0b817af5a8447af389/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:36.896993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2241ff1-456a-41c5-ab23-8c27caae110d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5091062794a94b248c26bbf81d987e30, user_id=89b2435ed2ca4d0b817af5a8447af389, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:36.899742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2241ff1-456a-41c5-ab23-8c27caae110d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:36.910338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1019/2041] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:36 2022] PUT /identity/v3/projects/5091062794a94b248c26bbf81d987e30/users/89b2435ed2ca4d0b817af5a8447af389/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:36.921858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40635f1c-8142-4d16-aa4a-8a50a45fe8a1 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:36.922145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40635f1c-8142-4d16-aa4a-8a50a45fe8a1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:36.922423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40635f1c-8142-4d16-aa4a-8a50a45fe8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:36.951283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-40635f1c-8142-4d16-aa4a-8a50a45fe8a1 None None] MFA Rules not processed for user `89b2435ed2ca4d0b817af5a8447af389`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:36.952526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-40635f1c-8142-4d16-aa4a-8a50a45fe8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.042104 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1020/2042] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:37.062119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40635f1c-8142-4d16-aa4a-8a50a45fe8a1 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:37.064354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-40635f1c-8142-4d16-aa4a-8a50a45fe8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.092703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83407d7e8c114a9c8290406f49891e38'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:37.093488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83407d7e8c114a9c8290406f49891e38'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:37.095133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83407d7e8c114a9c8290406f49891e38'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:37.122629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40635f1c-8142-4d16-aa4a-8a50a45fe8a1 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:37.127010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d6782e-0fc3-48be-a0fd-1405d31db3d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:37.127659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6782e-0fc3-48be-a0fd-1405d31db3d8 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:37.127994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6782e-0fc3-48be-a0fd-1405d31db3d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:37.128267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d6782e-0fc3-48be-a0fd-1405d31db3d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:37.130021 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b3d6782e-0fc3-48be-a0fd-1405d31db3d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.147390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6782e-0fc3-48be-a0fd-1405d31db3d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:37.150783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d6782e-0fc3-48be-a0fd-1405d31db3d8 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:37.160001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1021/2043] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:37 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:37.173247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83407d7e8c114a9c8290406f49891e38'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:37.199995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83407d7e8c114a9c8290406f49891e38'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:37.213495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1023/2044] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:36 2022] DELETE /identity/v3/users/83407d7e8c114a9c8290406f49891e38 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:37.217863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fb377-c46b-408f-a06a-756ca937bbe7 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:37.218155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fb377-c46b-408f-a06a-756ca937bbe7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:37.218428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7fb377-c46b-408f-a06a-756ca937bbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:37.237418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ac7fb377-c46b-408f-a06a-756ca937bbe7 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:37.238724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ac7fb377-c46b-408f-a06a-756ca937bbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.268805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:37.300855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1022/2045] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:37.310493 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:37.311970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.314477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fb377-c46b-408f-a06a-756ca937bbe7 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:37.315971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ac7fb377-c46b-408f-a06a-756ca937bbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.329026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7fb377-c46b-408f-a06a-756ca937bbe7 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:37.334866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:37.335819 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:37.336106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:37.336383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] PATH_INFO: `/v3/users/51a2672697024fd4821d1ad385b4daef` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:37.337596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51a2672697024fd4821d1ad385b4daef)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:37.340193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:37.364474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:37.380077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72697a3-d4ae-49be-8d90-dba207940d94 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:37.384082 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967a483-7521-4f16-926d-af21c659c777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:37.384709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967a483-7521-4f16-926d-af21c659c777 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:37.385330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967a483-7521-4f16-926d-af21c659c777 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:37.385794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967a483-7521-4f16-926d-af21c659c777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:37.396646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967a483-7521-4f16-926d-af21c659c777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:37.398068 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967a483-7521-4f16-926d-af21c659c777 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:37.408372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1024/2046] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:37.409222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967a483-7521-4f16-926d-af21c659c777 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:37.410549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c967a483-7521-4f16-926d-af21c659c777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.469255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967a483-7521-4f16-926d-af21c659c777 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:37.473358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89314419-fde0-4602-81bb-70d6ce69a562 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:37.473957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89314419-fde0-4602-81bb-70d6ce69a562 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:37.474327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89314419-fde0-4602-81bb-70d6ce69a562 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:37.474496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89314419-fde0-4602-81bb-70d6ce69a562 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:37.476332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-89314419-fde0-4602-81bb-70d6ce69a562 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.492907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51a2672697024fd4821d1ad385b4daef'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:37.493730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51a2672697024fd4821d1ad385b4daef'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:37.494477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51a2672697024fd4821d1ad385b4daef'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:37.538859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89314419-fde0-4602-81bb-70d6ce69a562 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:37.540316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89314419-fde0-4602-81bb-70d6ce69a562 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:37.550150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1025/2047] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:37.572881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51a2672697024fd4821d1ad385b4daef'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:37.589473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51a2672697024fd4821d1ad385b4daef'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:37.603054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1023/2048] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:37 2022] DELETE /identity/v3/users/51a2672697024fd4821d1ad385b4daef => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:37.607764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085566ea-321a-4206-8bb7-071d6492765f None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:37.608136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085566ea-321a-4206-8bb7-071d6492765f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:37.608500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085566ea-321a-4206-8bb7-071d6492765f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:37.628708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-085566ea-321a-4206-8bb7-071d6492765f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:37.630087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-085566ea-321a-4206-8bb7-071d6492765f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.696188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1026/2049] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:37.707382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:37.709081 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.774645 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13818204-a4fe-4b26-9ec2-adeccea33e01 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:37.778377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be6a7a-3959-4845-b7dd-2b2fca8b8c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:37.778997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be6a7a-3959-4845-b7dd-2b2fca8b8c54 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:37.779331 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be6a7a-3959-4845-b7dd-2b2fca8b8c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:37.779603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be6a7a-3959-4845-b7dd-2b2fca8b8c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:37.791429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be6a7a-3959-4845-b7dd-2b2fca8b8c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:37.792841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be6a7a-3959-4845-b7dd-2b2fca8b8c54 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:37.803332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1024/2050] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:37.848154 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085566ea-321a-4206-8bb7-071d6492765f None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:37.850023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-085566ea-321a-4206-8bb7-071d6492765f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:37.866321 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085566ea-321a-4206-8bb7-071d6492765f None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:37.872425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:37.873340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:37.873748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:37.874121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] PATH_INFO: `/v3/projects/82c7f9291b174fa09c4d85e891d986cf` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:37.876009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c7f9291b174fa09c4d85e891d986cf)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:37.878170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:37.997338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:38.011072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c7f9291b174fa09c4d85e891d986cf', 'request_id': 'req-c3f2235e-1631-4f76-9b6f-defee8101ec6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.026588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c7f9291b174fa09c4d85e891d986cf', 'request_id': 'req-c3f2235e-1631-4f76-9b6f-defee8101ec6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.032387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c7f9291b174fa09c4d85e891d986cf', 'request_id': 'req-c3f2235e-1631-4f76-9b6f-defee8101ec6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.110379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] The token cache is being invalidate because project 82c7f9291b174fa09c4d85e891d986cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:38.111013 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.111387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c3f2235e-1631-4f76-9b6f-defee8101ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.113599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1027/2051] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:37 2022] DELETE /identity/v3/projects/82c7f9291b174fa09c4d85e891d986cf => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:38.118218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7a2f9-f107-43fd-8e5a-3d2f1be7b202 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.118506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7a2f9-f107-43fd-8e5a-3d2f1be7b202 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.118850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7a2f9-f107-43fd-8e5a-3d2f1be7b202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.139897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-68a7a2f9-f107-43fd-8e5a-3d2f1be7b202 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:38.151085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-68a7a2f9-f107-43fd-8e5a-3d2f1be7b202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:38.195748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:38.212748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1025/2052] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:38.221617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d65a7-3f51-4017-90a2-19e366c69892 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.221907 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d65a7-3f51-4017-90a2-19e366c69892 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.222181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d65a7-3f51-4017-90a2-19e366c69892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.244889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-145d65a7-3f51-4017-90a2-19e366c69892 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:38.245525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-145d65a7-3f51-4017-90a2-19e366c69892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:38.246257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7a2f9-f107-43fd-8e5a-3d2f1be7b202 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:38.247999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-68a7a2f9-f107-43fd-8e5a-3d2f1be7b202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:38.254797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1028/2053] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:38.258832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d65a7-3f51-4017-90a2-19e366c69892 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:38.261230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7a2f9-f107-43fd-8e5a-3d2f1be7b202 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:38.267726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:38.268363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.268639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.268907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] PATH_INFO: `/v3/projects/24a51689c5d1404e8745688ad9ef3902` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.269966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d65a7-3f51-4017-90a2-19e366c69892 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:38.270081 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a51689c5d1404e8745688ad9ef3902)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:38.271978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:38.274822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd9911-78cd-4526-9c18-422d6cd5b374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:38.275544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd9911-78cd-4526-9c18-422d6cd5b374 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.275886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd9911-78cd-4526-9c18-422d6cd5b374 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.276250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd9911-78cd-4526-9c18-422d6cd5b374 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.276956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd9911-78cd-4526-9c18-422d6cd5b374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:01:38.277318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd9911-78cd-4526-9c18-422d6cd5b374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:38.278937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd9911-78cd-4526-9c18-422d6cd5b374 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:38.292724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1029/2054] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:01:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:01:38.296983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd9911-78cd-4526-9c18-422d6cd5b374 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:38.308662 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd9911-78cd-4526-9c18-422d6cd5b374 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:38.313819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e2e8c-0323-4549-af50-a5a4c98a0118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:38.314440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e2e8c-0323-4549-af50-a5a4c98a0118 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.314718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e2e8c-0323-4549-af50-a5a4c98a0118 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.314986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e2e8c-0323-4549-af50-a5a4c98a0118 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.315631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e2e8c-0323-4549-af50-a5a4c98a0118 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:38.317509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e2e8c-0323-4549-af50-a5a4c98a0118 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:38.401107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a51689c5d1404e8745688ad9ef3902', 'request_id': 'req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.407641 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a51689c5d1404e8745688ad9ef3902', 'request_id': 'req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.415002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1030/2055] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:38 2022] POST /identity/v3/projects => generated 374 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:38.418951 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e2e8c-0323-4549-af50-a5a4c98a0118 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:38.480903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e2e8c-0323-4549-af50-a5a4c98a0118 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:38.486052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8f690-09b2-447c-809a-1a2618c0cd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:38.486654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f690-09b2-447c-809a-1a2618c0cd12 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.486933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f690-09b2-447c-809a-1a2618c0cd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.487235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f690-09b2-447c-809a-1a2618c0cd12 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.487333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a51689c5d1404e8745688ad9ef3902', 'request_id': 'req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.487869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8f690-09b2-447c-809a-1a2618c0cd12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:38.489800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8f690-09b2-447c-809a-1a2618c0cd12 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:38.504695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] The token cache is being invalidate because project 24a51689c5d1404e8745688ad9ef3902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:38.505216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.505847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.507438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1026/2056] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:38 2022] DELETE /identity/v3/projects/24a51689c5d1404e8745688ad9ef3902 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:38.511214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:38.514556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:38.562783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1031/2057] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:38 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:38.566853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8f690-09b2-447c-809a-1a2618c0cd12 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:38.568604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-98a8f690-09b2-447c-809a-1a2618c0cd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:38.579339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a204a2-2d7b-40dd-b5d0-5497f72379f4 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:38.581037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8f690-09b2-447c-809a-1a2618c0cd12 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:38.584433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:38.585085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.585362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.585631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] PATH_INFO: `/v3/users/6d578e457a6d4576a6c3a2d72a5ce197` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.585866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261b88c-f237-4a87-8162-d5a0e0131242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:38.586463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261b88c-f237-4a87-8162-d5a0e0131242 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.586741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261b88c-f237-4a87-8162-d5a0e0131242 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.586904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d578e457a6d4576a6c3a2d72a5ce197)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:38.587007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261b88c-f237-4a87-8162-d5a0e0131242 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.587739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261b88c-f237-4a87-8162-d5a0e0131242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:38.588849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:38.595552 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261b88c-f237-4a87-8162-d5a0e0131242 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:38.600839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1032/2058] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:38 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:01:38.604915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261b88c-f237-4a87-8162-d5a0e0131242 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:38.614936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261b88c-f237-4a87-8162-d5a0e0131242 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:38.619984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9c860-826a-4111-bce0-6a1f0566a173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:38.620632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9c860-826a-4111-bce0-6a1f0566a173 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.620913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9c860-826a-4111-bce0-6a1f0566a173 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.621186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9c860-826a-4111-bce0-6a1f0566a173 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.621817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9c860-826a-4111-bce0-6a1f0566a173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:38.623436 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9c860-826a-4111-bce0-6a1f0566a173 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:38.635076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1033/2059] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:38 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:01:38.639273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9c860-826a-4111-bce0-6a1f0566a173 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:38.649612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9c860-826a-4111-bce0-6a1f0566a173 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:38.654266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372daf1d-4a98-48fb-8243-c8eeab24cf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:38.654888 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372daf1d-4a98-48fb-8243-c8eeab24cf76 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.655233 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372daf1d-4a98-48fb-8243-c8eeab24cf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.655510 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372daf1d-4a98-48fb-8243-c8eeab24cf76 admin admin] PATH_INFO: `/v3/projects/4f3f7cf061114254b3a440f64b16d803/users/ebea1b500dd44a9888d5cd18c55b0909/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.669569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372daf1d-4a98-48fb-8243-c8eeab24cf76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3f7cf061114254b3a440f64b16d803, user_id=ebea1b500dd44a9888d5cd18c55b0909, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:38.672266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372daf1d-4a98-48fb-8243-c8eeab24cf76 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:38.773597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1034/2060] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:38 2022] PUT /identity/v3/projects/4f3f7cf061114254b3a440f64b16d803/users/ebea1b500dd44a9888d5cd18c55b0909/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:38.784722 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4466a2-535d-4131-b747-631a8f8ac686 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:38.785008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4466a2-535d-4131-b747-631a8f8ac686 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:38.785283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4466a2-535d-4131-b747-631a8f8ac686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:38.813316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-db4466a2-535d-4131-b747-631a8f8ac686 None None] MFA Rules not processed for user `ebea1b500dd44a9888d5cd18c55b0909`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:38.814526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-db4466a2-535d-4131-b747-631a8f8ac686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:38.822821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:38.832604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d578e457a6d4576a6c3a2d72a5ce197'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.833254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d578e457a6d4576a6c3a2d72a5ce197'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.833870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d578e457a6d4576a6c3a2d72a5ce197'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.878802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1035/2061] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:38.896075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4466a2-535d-4131-b747-631a8f8ac686 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:38.897478 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-db4466a2-535d-4131-b747-631a8f8ac686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:38.910309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d578e457a6d4576a6c3a2d72a5ce197'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.925536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d578e457a6d4576a6c3a2d72a5ce197'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:38.928146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:38.940305 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1027/2062] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:38 2022] DELETE /identity/v3/users/6d578e457a6d4576a6c3a2d72a5ce197 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:38.987355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4466a2-535d-4131-b747-631a8f8ac686 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:39.053864 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70618e-342a-4651-8633-31c5553c0c97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:39.054630 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70618e-342a-4651-8633-31c5553c0c97 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:39.055080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70618e-342a-4651-8633-31c5553c0c97 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:39.055454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70618e-342a-4651-8633-31c5553c0c97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:39.118655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70618e-342a-4651-8633-31c5553c0c97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:39.120177 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70618e-342a-4651-8633-31c5553c0c97 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:39.131219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1036/2063] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:39.391799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:39.393459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:39.471265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16715533-4251-472c-9d14-4ed69a10c992 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:39.475998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:39.476625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:39.476902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:39.477174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] PATH_INFO: `/v3/projects/76758dbeb43241a98659aef117c529ed` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:39.478313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=76758dbeb43241a98659aef117c529ed)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:39.480628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:39.628361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76758dbeb43241a98659aef117c529ed', 'request_id': 'req-e55a9018-fc00-4b6d-85c3-752399f5a6ff'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:39.635985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76758dbeb43241a98659aef117c529ed', 'request_id': 'req-e55a9018-fc00-4b6d-85c3-752399f5a6ff'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:39.728636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76758dbeb43241a98659aef117c529ed', 'request_id': 'req-e55a9018-fc00-4b6d-85c3-752399f5a6ff'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:39.740510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] The token cache is being invalidate because project 76758dbeb43241a98659aef117c529ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:39.740798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:39.741437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e55a9018-fc00-4b6d-85c3-752399f5a6ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:39.742962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1028/2064] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:39 2022] DELETE /identity/v3/projects/76758dbeb43241a98659aef117c529ed => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:39.747969 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccca882-c620-4661-9e24-9650f3890f61 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:39.748262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccca882-c620-4661-9e24-9650f3890f61 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:39.748562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccca882-c620-4661-9e24-9650f3890f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:39.769450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-cccca882-c620-4661-9e24-9650f3890f61 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:39.770802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cccca882-c620-4661-9e24-9650f3890f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:39.787032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:39.866143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1037/2065] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:39.867879 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f94a0-b846-49ea-b1a6-41433dbb3066 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:39.868093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f94a0-b846-49ea-b1a6-41433dbb3066 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:39.868369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429f94a0-b846-49ea-b1a6-41433dbb3066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:39.894960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-429f94a0-b846-49ea-b1a6-41433dbb3066 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:39.896370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-429f94a0-b846-49ea-b1a6-41433dbb3066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:39.907352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1029/2066] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:39.911871 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccca882-c620-4661-9e24-9650f3890f61 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:39.922534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccca882-c620-4661-9e24-9650f3890f61 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:39.927999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1711e0-c241-4632-a885-eea052602390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:39.929415 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1711e0-c241-4632-a885-eea052602390 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:39.929415 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1711e0-c241-4632-a885-eea052602390 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:39.929415 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1711e0-c241-4632-a885-eea052602390 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:39.930296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1711e0-c241-4632-a885-eea052602390 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:01:39.930788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1711e0-c241-4632-a885-eea052602390 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:39.932543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1711e0-c241-4632-a885-eea052602390 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:39.953947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1038/2067] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:01:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:01:39.959002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f94a0-b846-49ea-b1a6-41433dbb3066 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:39.976476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429f94a0-b846-49ea-b1a6-41433dbb3066 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:39.976476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48457c-0299-4269-97e4-e9f84a9e2219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:39.976476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48457c-0299-4269-97e4-e9f84a9e2219 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:39.976921 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48457c-0299-4269-97e4-e9f84a9e2219 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:39.977259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48457c-0299-4269-97e4-e9f84a9e2219 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:39.977835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48457c-0299-4269-97e4-e9f84a9e2219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:39.979810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48457c-0299-4269-97e4-e9f84a9e2219 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:40.092963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1030/2068] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:39 2022] POST /identity/v3/projects => generated 390 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:40.094200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1711e0-c241-4632-a885-eea052602390 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:40.105187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1711e0-c241-4632-a885-eea052602390 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:40.167161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5f80f-8c3f-4c1b-9650-24a34aac0e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:40.167848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5f80f-8c3f-4c1b-9650-24a34aac0e24 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:40.168043 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5f80f-8c3f-4c1b-9650-24a34aac0e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:40.168323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5f80f-8c3f-4c1b-9650-24a34aac0e24 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:40.168957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5f80f-8c3f-4c1b-9650-24a34aac0e24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:40.170888 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5f80f-8c3f-4c1b-9650-24a34aac0e24 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:40.274392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1039/2069] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:40 2022] POST /identity/v3/users => generated 296 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:40.278585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48457c-0299-4269-97e4-e9f84a9e2219 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:40.292957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48457c-0299-4269-97e4-e9f84a9e2219 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:40.298055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626141ea-8ae9-4a89-bd40-6ef3828c5669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:40.298961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626141ea-8ae9-4a89-bd40-6ef3828c5669 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:40.299282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626141ea-8ae9-4a89-bd40-6ef3828c5669 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:40.299552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626141ea-8ae9-4a89-bd40-6ef3828c5669 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:40.300361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626141ea-8ae9-4a89-bd40-6ef3828c5669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:40.301962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626141ea-8ae9-4a89-bd40-6ef3828c5669 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:40.313075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1031/2070] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:40 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:01:40.317518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5f80f-8c3f-4c1b-9650-24a34aac0e24 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:40.328002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5f80f-8c3f-4c1b-9650-24a34aac0e24 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:40.332737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f2482-218c-46b3-bb1c-00aecad1d39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:40.333389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f2482-218c-46b3-bb1c-00aecad1d39e admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:40.333674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f2482-218c-46b3-bb1c-00aecad1d39e admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:40.333948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f2482-218c-46b3-bb1c-00aecad1d39e admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:40.334587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f2482-218c-46b3-bb1c-00aecad1d39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:40.336234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f2482-218c-46b3-bb1c-00aecad1d39e admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:40.348417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1040/2071] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:40 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:40.352729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626141ea-8ae9-4a89-bd40-6ef3828c5669 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:40.363654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626141ea-8ae9-4a89-bd40-6ef3828c5669 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:40.369094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948ecd5-bc03-470a-8128-5b95966c82a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:40.369760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ecd5-bc03-470a-8128-5b95966c82a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:40.370039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ecd5-bc03-470a-8128-5b95966c82a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:40.370332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ecd5-bc03-470a-8128-5b95966c82a9 admin admin] PATH_INFO: `/v3/projects/ce49ca0f2c0d43368f9b998048eaed18/users/13eef12da39b43728101c66a919402df/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:40.386516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948ecd5-bc03-470a-8128-5b95966c82a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce49ca0f2c0d43368f9b998048eaed18, user_id=13eef12da39b43728101c66a919402df, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:40.389299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948ecd5-bc03-470a-8128-5b95966c82a9 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:40.494370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1032/2072] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:40 2022] PUT /identity/v3/projects/ce49ca0f2c0d43368f9b998048eaed18/users/13eef12da39b43728101c66a919402df/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:40.509981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc9960-5527-4600-8c43-ef57ee480908 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:40.519338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc9960-5527-4600-8c43-ef57ee480908 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:40.519338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc9960-5527-4600-8c43-ef57ee480908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:40.544832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b2dc9960-5527-4600-8c43-ef57ee480908 None None] MFA Rules not processed for user `13eef12da39b43728101c66a919402df`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:40.546067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b2dc9960-5527-4600-8c43-ef57ee480908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:40.650802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1041/2073] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:40 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:40.651860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948ecd5-bc03-470a-8128-5b95966c82a9 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:40.685236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:40.732160 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948ecd5-bc03-470a-8128-5b95966c82a9 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:40.738367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf986f0-6311-4b8f-8031-adc097cc652f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:40.739226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf986f0-6311-4b8f-8031-adc097cc652f admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:40.739580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf986f0-6311-4b8f-8031-adc097cc652f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:40.739919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf986f0-6311-4b8f-8031-adc097cc652f admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:40.740765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf986f0-6311-4b8f-8031-adc097cc652f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:40.742748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf986f0-6311-4b8f-8031-adc097cc652f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:40.833786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1033/2074] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:40 2022] POST /identity/v3/projects => generated 390 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:40.837695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc9960-5527-4600-8c43-ef57ee480908 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:40.848852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc9960-5527-4600-8c43-ef57ee480908 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:40.861832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:40.924284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751248-1668-49c3-85be-1de60e0d5dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:40.925236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751248-1668-49c3-85be-1de60e0d5dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:40.925535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751248-1668-49c3-85be-1de60e0d5dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:40.925811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3751248-1668-49c3-85be-1de60e0d5dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:40.926995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3751248-1668-49c3-85be-1de60e0d5dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:40.929095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3751248-1668-49c3-85be-1de60e0d5dc0 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:40.954642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e9174-03e7-4276-b973-13389d2ee6fb None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:40.954953 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e9174-03e7-4276-b973-13389d2ee6fb None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:40.955264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e9174-03e7-4276-b973-13389d2ee6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:40.975793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-753e9174-03e7-4276-b973-13389d2ee6fb None None] MFA Rules not processed for user `ebea1b500dd44a9888d5cd18c55b0909`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:40.976992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-753e9174-03e7-4276-b973-13389d2ee6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:41.035179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1034/2075] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:41.054929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1042/2076] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:40 2022] POST /identity/v3/users => generated 302 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:41.058592 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e9174-03e7-4276-b973-13389d2ee6fb None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:41.070913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e9174-03e7-4276-b973-13389d2ee6fb None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:41.075722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14f4e6-7181-4851-9c96-c7f30dd7d5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:41.076474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14f4e6-7181-4851-9c96-c7f30dd7d5fc admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:41.076980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14f4e6-7181-4851-9c96-c7f30dd7d5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:41.077315 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f14f4e6-7181-4851-9c96-c7f30dd7d5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:41.078181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f14f4e6-7181-4851-9c96-c7f30dd7d5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:41.079994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f14f4e6-7181-4851-9c96-c7f30dd7d5fc admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:41.092423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1035/2077] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:41 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:41.097294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751248-1668-49c3-85be-1de60e0d5dc0 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:41.112339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3751248-1668-49c3-85be-1de60e0d5dc0 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:41.117664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74f346-ec4a-4192-9aee-7649e5327ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:41.118322 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74f346-ec4a-4192-9aee-7649e5327ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:41.118597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74f346-ec4a-4192-9aee-7649e5327ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:41.118986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c74f346-ec4a-4192-9aee-7649e5327ef8 admin admin] PATH_INFO: `/v3/projects/f50bade8bbf0440e893ec36b1a8ac1bb/users/99e2f36c54f341cbba620308e36cd02e/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:41.135589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74f346-ec4a-4192-9aee-7649e5327ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50bade8bbf0440e893ec36b1a8ac1bb, user_id=99e2f36c54f341cbba620308e36cd02e, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:41.138265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c74f346-ec4a-4192-9aee-7649e5327ef8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:41.199882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1043/2078] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:41 2022] PUT /identity/v3/projects/f50bade8bbf0440e893ec36b1a8ac1bb/users/99e2f36c54f341cbba620308e36cd02e/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:41.205924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14f4e6-7181-4851-9c96-c7f30dd7d5fc admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:41.216393 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f14f4e6-7181-4851-9c96-c7f30dd7d5fc admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:41.312629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa49347-cbc2-4741-8d9a-fea23f8e445f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:41.313325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa49347-cbc2-4741-8d9a-fea23f8e445f admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:41.313611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa49347-cbc2-4741-8d9a-fea23f8e445f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:41.314585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa49347-cbc2-4741-8d9a-fea23f8e445f admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:41.315151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa49347-cbc2-4741-8d9a-fea23f8e445f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:41.317262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa49347-cbc2-4741-8d9a-fea23f8e445f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:41.333161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1036/2079] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:41 2022] GET /identity/v3/roles => generated 1427 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:41.336380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74f346-ec4a-4192-9aee-7649e5327ef8 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:41.354439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c74f346-ec4a-4192-9aee-7649e5327ef8 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:41.373180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85b759-4cf1-4af8-b085-a4102ad615d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:41.376519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85b759-4cf1-4af8-b085-a4102ad615d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:41.378846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85b759-4cf1-4af8-b085-a4102ad615d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:41.384920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85b759-4cf1-4af8-b085-a4102ad615d3 admin admin] PATH_INFO: `/v3/projects/f50bade8bbf0440e893ec36b1a8ac1bb/users/99e2f36c54f341cbba620308e36cd02e/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:41.386625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85b759-4cf1-4af8-b085-a4102ad615d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f50bade8bbf0440e893ec36b1a8ac1bb, user_id=99e2f36c54f341cbba620308e36cd02e, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:41.393815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85b759-4cf1-4af8-b085-a4102ad615d3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:41.404981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1044/2080] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:41 2022] PUT /identity/v3/projects/f50bade8bbf0440e893ec36b1a8ac1bb/users/99e2f36c54f341cbba620308e36cd02e/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:41.416260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17332fc-2ae1-4cfa-ac53-58a88e6a2f89 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:41.416637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17332fc-2ae1-4cfa-ac53-58a88e6a2f89 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:41.416979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17332fc-2ae1-4cfa-ac53-58a88e6a2f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:41.453548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-a17332fc-2ae1-4cfa-ac53-58a88e6a2f89 None None] MFA Rules not processed for user `99e2f36c54f341cbba620308e36cd02e`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:41.455273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a17332fc-2ae1-4cfa-ac53-58a88e6a2f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:41.565320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1037/2081] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:41 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:41.577037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85b759-4cf1-4af8-b085-a4102ad615d3 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:41.579229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-bd85b759-4cf1-4af8-b085-a4102ad615d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:41.638676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85b759-4cf1-4af8-b085-a4102ad615d3 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:41.642509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff5fcc-6a54-4daa-8fc0-0e7cebd4cc62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:41.643139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff5fcc-6a54-4daa-8fc0-0e7cebd4cc62 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:41.643416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff5fcc-6a54-4daa-8fc0-0e7cebd4cc62 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:41.643684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff5fcc-6a54-4daa-8fc0-0e7cebd4cc62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:41.652692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff5fcc-6a54-4daa-8fc0-0e7cebd4cc62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:41.654214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff5fcc-6a54-4daa-8fc0-0e7cebd4cc62 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:41.665098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1045/2082] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:41 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:41.785277 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:41.833168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17332fc-2ae1-4cfa-ac53-58a88e6a2f89 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:41.904705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17332fc-2ae1-4cfa-ac53-58a88e6a2f89 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:41.909612 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:41.910243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:41.910520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:41.910787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] PATH_INFO: `/v3/users/13eef12da39b43728101c66a919402df` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:41.911961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=13eef12da39b43728101c66a919402df)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:41.913897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:42.141804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13eef12da39b43728101c66a919402df'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:42.142464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13eef12da39b43728101c66a919402df'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:42.143141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13eef12da39b43728101c66a919402df'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:42.212715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13eef12da39b43728101c66a919402df'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:42.225606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13eef12da39b43728101c66a919402df'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:42.237304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1038/2083] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:41 2022] DELETE /identity/v3/users/13eef12da39b43728101c66a919402df => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:42.241843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f75893-5171-4b3e-a766-c68d6674fc03 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:42.242296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f75893-5171-4b3e-a766-c68d6674fc03 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:42.242409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f75893-5171-4b3e-a766-c68d6674fc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:42.260911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-a1f75893-5171-4b3e-a766-c68d6674fc03 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:42.262119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a1f75893-5171-4b3e-a766-c68d6674fc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:42.334372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1046/2084] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:42.344120 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:42.345558 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:42.403409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89f682-56f6-4195-bcfb-ff55ad55dbcc admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:42.407384 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588e562-310c-42f2-b845-cfe79bee862c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:42.407982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588e562-310c-42f2-b845-cfe79bee862c service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:42.408258 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588e562-310c-42f2-b845-cfe79bee862c service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:42.408527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4588e562-310c-42f2-b845-cfe79bee862c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:42.417655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588e562-310c-42f2-b845-cfe79bee862c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:42.419080 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4588e562-310c-42f2-b845-cfe79bee862c service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:42.429076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1039/2085] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:42.916372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:42.925090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f75893-5171-4b3e-a766-c68d6674fc03 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:42.927898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a1f75893-5171-4b3e-a766-c68d6674fc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:42.963462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f75893-5171-4b3e-a766-c68d6674fc03 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:42.971111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:42.971911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:42.972363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:42.972783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] PATH_INFO: `/v3/projects/ce49ca0f2c0d43368f9b998048eaed18` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:42.974458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce49ca0f2c0d43368f9b998048eaed18)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:42.976613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:42.989008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:43.134502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce49ca0f2c0d43368f9b998048eaed18', 'request_id': 'req-8b6de699-72df-4696-a4e7-8b6805a49af1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:43.142724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce49ca0f2c0d43368f9b998048eaed18', 'request_id': 'req-8b6de699-72df-4696-a4e7-8b6805a49af1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:43.148405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce49ca0f2c0d43368f9b998048eaed18', 'request_id': 'req-8b6de699-72df-4696-a4e7-8b6805a49af1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:43.264337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] The token cache is being invalidate because project ce49ca0f2c0d43368f9b998048eaed18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:43.266246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:43.267267 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:43.268955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1047/2086] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:42 2022] DELETE /identity/v3/projects/ce49ca0f2c0d43368f9b998048eaed18 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:43.273060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588e562-310c-42f2-b845-cfe79bee862c service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:43.282989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4588e562-310c-42f2-b845-cfe79bee862c service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:43.338527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:43.339186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:43.339468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:43.339739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] PATH_INFO: `/v3/users/99e2f36c54f341cbba620308e36cd02e` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:43.340854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e2f36c54f341cbba620308e36cd02e)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:43.342754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:43.490913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e2f36c54f341cbba620308e36cd02e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:43.493377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e2f36c54f341cbba620308e36cd02e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:43.493377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e2f36c54f341cbba620308e36cd02e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:43.543993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e2f36c54f341cbba620308e36cd02e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:43.557486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c2e3d688-b17d-403f-a479-756748b79863 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e2f36c54f341cbba620308e36cd02e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:43.570129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1040/2087] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:43 2022] DELETE /identity/v3/users/99e2f36c54f341cbba620308e36cd02e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:44.020392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:44.021846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:44.034854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6de699-72df-4696-a4e7-8b6805a49af1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:44.092791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:44.093457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:44.093735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:44.093993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] PATH_INFO: `/v3/projects/f50bade8bbf0440e893ec36b1a8ac1bb` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:44.095164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f50bade8bbf0440e893ec36b1a8ac1bb)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:44.097071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:44.218911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f50bade8bbf0440e893ec36b1a8ac1bb', 'request_id': 'req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:44.225424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f50bade8bbf0440e893ec36b1a8ac1bb', 'request_id': 'req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:44.231126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f50bade8bbf0440e893ec36b1a8ac1bb', 'request_id': 'req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:44.343919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] The token cache is being invalidate because project f50bade8bbf0440e893ec36b1a8ac1bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:44.344525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:44.344770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-69bcfa8c-3ffb-4c84-ac48-e1702d4fcb98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:44.346251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1048/2088] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:44 2022] DELETE /identity/v3/projects/f50bade8bbf0440e893ec36b1a8ac1bb => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:44.351165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6443f-117b-420e-bc49-346e882dc955 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:44.351589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6443f-117b-420e-bc49-346e882dc955 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:44.351702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6443f-117b-420e-bc49-346e882dc955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:44.371497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b9f6443f-117b-420e-bc49-346e882dc955 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:44.372692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b9f6443f-117b-420e-bc49-346e882dc955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:44.426126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:44.434430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1041/2089] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:44.443458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcab8e-a8ee-48f9-8752-540a1b8b28d6 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:44.443740 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcab8e-a8ee-48f9-8752-540a1b8b28d6 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:44.444010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bcab8e-a8ee-48f9-8752-540a1b8b28d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:44.463849 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-30bcab8e-a8ee-48f9-8752-540a1b8b28d6 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:44.465087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-30bcab8e-a8ee-48f9-8752-540a1b8b28d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:44.473965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1049/2090] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:44.478205 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6443f-117b-420e-bc49-346e882dc955 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:44.487587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6443f-117b-420e-bc49-346e882dc955 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:44.492715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb1b5a-a269-47b7-b3d5-fba8b8f1e375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:44.493339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb1b5a-a269-47b7-b3d5-fba8b8f1e375 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:44.493642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb1b5a-a269-47b7-b3d5-fba8b8f1e375 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:44.493927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb1b5a-a269-47b7-b3d5-fba8b8f1e375 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:44.494552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb1b5a-a269-47b7-b3d5-fba8b8f1e375 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:01:44.494850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb1b5a-a269-47b7-b3d5-fba8b8f1e375 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:44.496491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb1b5a-a269-47b7-b3d5-fba8b8f1e375 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:44.512088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1042/2091] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:01:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:01:44.513450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcab8e-a8ee-48f9-8752-540a1b8b28d6 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:44.523519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bcab8e-a8ee-48f9-8752-540a1b8b28d6 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:44.528186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820dcf7-36ba-41a8-a0e8-382945c67d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:44.528769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820dcf7-36ba-41a8-a0e8-382945c67d9a admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:44.529045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820dcf7-36ba-41a8-a0e8-382945c67d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:44.529315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820dcf7-36ba-41a8-a0e8-382945c67d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:44.529934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820dcf7-36ba-41a8-a0e8-382945c67d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:44.532852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820dcf7-36ba-41a8-a0e8-382945c67d9a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:44.549569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:44.632430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1050/2092] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:44 2022] POST /identity/v3/projects => generated 386 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:44.636714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb1b5a-a269-47b7-b3d5-fba8b8f1e375 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:44.647177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb1b5a-a269-47b7-b3d5-fba8b8f1e375 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:44.703836 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a6be2-838d-46da-b8cf-63fbee99165d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:44.704437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a6be2-838d-46da-b8cf-63fbee99165d admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:44.704735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a6be2-838d-46da-b8cf-63fbee99165d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:44.705003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20a6be2-838d-46da-b8cf-63fbee99165d admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:44.705625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a6be2-838d-46da-b8cf-63fbee99165d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:44.708415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20a6be2-838d-46da-b8cf-63fbee99165d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:44.843756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1043/2093] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:44 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:44.847919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820dcf7-36ba-41a8-a0e8-382945c67d9a admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:44.858625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820dcf7-36ba-41a8-a0e8-382945c67d9a admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:44.863947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29a264-cdc8-47ce-99fa-023ce5ebfd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:44.864571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a264-cdc8-47ce-99fa-023ce5ebfd48 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:44.865100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a264-cdc8-47ce-99fa-023ce5ebfd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:44.865394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a264-cdc8-47ce-99fa-023ce5ebfd48 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:44.866929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29a264-cdc8-47ce-99fa-023ce5ebfd48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:44.868628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29a264-cdc8-47ce-99fa-023ce5ebfd48 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:44.880276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1051/2094] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:44 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:44.888437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a6be2-838d-46da-b8cf-63fbee99165d admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:44.899592 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20a6be2-838d-46da-b8cf-63fbee99165d admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:44.904700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e479ca-15cf-4be9-8e8d-1a58abc6d43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:44.905313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e479ca-15cf-4be9-8e8d-1a58abc6d43e admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:44.913199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e479ca-15cf-4be9-8e8d-1a58abc6d43e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:44.913199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e479ca-15cf-4be9-8e8d-1a58abc6d43e admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:44.913199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e479ca-15cf-4be9-8e8d-1a58abc6d43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:44.913199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e479ca-15cf-4be9-8e8d-1a58abc6d43e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:44.926290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1044/2095] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:44 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:44.930441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29a264-cdc8-47ce-99fa-023ce5ebfd48 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:44.941621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29a264-cdc8-47ce-99fa-023ce5ebfd48 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:44.946305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ba5b4-cd0c-4e77-8a08-0f530eeadff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:44.947105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ba5b4-cd0c-4e77-8a08-0f530eeadff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:44.947389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ba5b4-cd0c-4e77-8a08-0f530eeadff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:44.947682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ba5b4-cd0c-4e77-8a08-0f530eeadff1 admin admin] PATH_INFO: `/v3/projects/33cab39ee8ff4c1fb904c60e04916fdb/users/228e7168799543eb8ab1ac79ed79559f/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:44.964083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ba5b4-cd0c-4e77-8a08-0f530eeadff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cab39ee8ff4c1fb904c60e04916fdb, user_id=228e7168799543eb8ab1ac79ed79559f, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:44.966769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ba5b4-cd0c-4e77-8a08-0f530eeadff1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:45.071150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1052/2096] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:44 2022] PUT /identity/v3/projects/33cab39ee8ff4c1fb904c60e04916fdb/users/228e7168799543eb8ab1ac79ed79559f/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:45.081773 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa000301-69fe-489e-90b2-1b1f2c8c9440 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:45.082056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa000301-69fe-489e-90b2-1b1f2c8c9440 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:45.082332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa000301-69fe-489e-90b2-1b1f2c8c9440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:45.123976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-fa000301-69fe-489e-90b2-1b1f2c8c9440 None None] MFA Rules not processed for user `228e7168799543eb8ab1ac79ed79559f`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:45.126086 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-fa000301-69fe-489e-90b2-1b1f2c8c9440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:45.220284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1045/2097] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:45 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:45.225134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ba5b4-cd0c-4e77-8a08-0f530eeadff1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:45.294683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ba5b4-cd0c-4e77-8a08-0f530eeadff1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:45.299929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84e9cf-e3cc-4d07-abad-f64f91143009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:45.300546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84e9cf-e3cc-4d07-abad-f64f91143009 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:45.300829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84e9cf-e3cc-4d07-abad-f64f91143009 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:45.301101 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84e9cf-e3cc-4d07-abad-f64f91143009 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:45.301749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84e9cf-e3cc-4d07-abad-f64f91143009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:45.303687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84e9cf-e3cc-4d07-abad-f64f91143009 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:45.411330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1053/2098] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:45 2022] POST /identity/v3/projects => generated 386 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:45.414458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa000301-69fe-489e-90b2-1b1f2c8c9440 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:45.430040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa000301-69fe-489e-90b2-1b1f2c8c9440 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:45.501845 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:45.511130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315d4c4-983f-40ba-a4bf-00b8e42af019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:45.513194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315d4c4-983f-40ba-a4bf-00b8e42af019 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:45.513194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315d4c4-983f-40ba-a4bf-00b8e42af019 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:45.513194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315d4c4-983f-40ba-a4bf-00b8e42af019 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:45.513682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315d4c4-983f-40ba-a4bf-00b8e42af019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:45.516463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315d4c4-983f-40ba-a4bf-00b8e42af019 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:45.619322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1046/2099] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:45 2022] POST /identity/v3/users => generated 300 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:01:45.623474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84e9cf-e3cc-4d07-abad-f64f91143009 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:45.635048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84e9cf-e3cc-4d07-abad-f64f91143009 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:45.639769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91414b-1e8a-4249-ab17-3be405404716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:45.641172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91414b-1e8a-4249-ab17-3be405404716 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:45.641449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91414b-1e8a-4249-ab17-3be405404716 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:45.641727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91414b-1e8a-4249-ab17-3be405404716 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:45.642404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91414b-1e8a-4249-ab17-3be405404716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:45.644221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91414b-1e8a-4249-ab17-3be405404716 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:45.655366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1054/2100] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:45 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:45.659578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315d4c4-983f-40ba-a4bf-00b8e42af019 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:45.669351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315d4c4-983f-40ba-a4bf-00b8e42af019 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:45.675086 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951c4e8-3aeb-407a-8877-cc2a2710c3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:45.675873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951c4e8-3aeb-407a-8877-cc2a2710c3bd admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:45.676094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951c4e8-3aeb-407a-8877-cc2a2710c3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:45.676285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0951c4e8-3aeb-407a-8877-cc2a2710c3bd admin admin] PATH_INFO: `/v3/projects/92e7fd26fa1e4af8ab2febaec4a866d5/users/ceee9d9a9301460880f0a4bb6f82a5ee/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:45.694639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951c4e8-3aeb-407a-8877-cc2a2710c3bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e7fd26fa1e4af8ab2febaec4a866d5, user_id=ceee9d9a9301460880f0a4bb6f82a5ee, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:45.697665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0951c4e8-3aeb-407a-8877-cc2a2710c3bd admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:45.821522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1047/2101] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:45 2022] PUT /identity/v3/projects/92e7fd26fa1e4af8ab2febaec4a866d5/users/ceee9d9a9301460880f0a4bb6f82a5ee/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:45.825525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91414b-1e8a-4249-ab17-3be405404716 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:45.836846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91414b-1e8a-4249-ab17-3be405404716 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:45.898521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b14ff-4ac8-46ef-8c7f-47f50932443d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:45.899215 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b14ff-4ac8-46ef-8c7f-47f50932443d admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:45.899492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b14ff-4ac8-46ef-8c7f-47f50932443d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:45.899760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b14ff-4ac8-46ef-8c7f-47f50932443d admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:45.900404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b14ff-4ac8-46ef-8c7f-47f50932443d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:45.902032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b14ff-4ac8-46ef-8c7f-47f50932443d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:45.916014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1055/2102] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:45 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:45.917105 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951c4e8-3aeb-407a-8877-cc2a2710c3bd admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:45.929478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0951c4e8-3aeb-407a-8877-cc2a2710c3bd admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:45.934154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7fb1b-4dad-45e9-b5a1-9df68429784b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:45.934838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7fb1b-4dad-45e9-b5a1-9df68429784b admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:45.935148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7fb1b-4dad-45e9-b5a1-9df68429784b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:45.935424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7fb1b-4dad-45e9-b5a1-9df68429784b admin admin] PATH_INFO: `/v3/projects/92e7fd26fa1e4af8ab2febaec4a866d5/users/ceee9d9a9301460880f0a4bb6f82a5ee/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:45.937939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7fb1b-4dad-45e9-b5a1-9df68429784b admin admin] RBAC: Authorizing `identity:create_grant(project_id=92e7fd26fa1e4af8ab2febaec4a866d5, user_id=ceee9d9a9301460880f0a4bb6f82a5ee, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:45.940781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7fb1b-4dad-45e9-b5a1-9df68429784b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:45.958423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1048/2103] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:45 2022] PUT /identity/v3/projects/92e7fd26fa1e4af8ab2febaec4a866d5/users/ceee9d9a9301460880f0a4bb6f82a5ee/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:45.971955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4efc7-6d34-432e-a9e9-42d65c455bce None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:45.972320 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4efc7-6d34-432e-a9e9-42d65c455bce None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:45.972655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4efc7-6d34-432e-a9e9-42d65c455bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:46.012516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f8c4efc7-6d34-432e-a9e9-42d65c455bce None None] MFA Rules not processed for user `ceee9d9a9301460880f0a4bb6f82a5ee`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:46.014133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f8c4efc7-6d34-432e-a9e9-42d65c455bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:46.033939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:46.092064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1056/2104] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:45 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:46.106888 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7fb1b-4dad-45e9-b5a1-9df68429784b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:46.110369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-44d7fb1b-4dad-45e9-b5a1-9df68429784b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:46.185781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7fb1b-4dad-45e9-b5a1-9df68429784b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:46.192577 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6738d-79df-4945-b8db-8cfeaef41bc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:46.193334 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6738d-79df-4945-b8db-8cfeaef41bc5 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:46.194170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6738d-79df-4945-b8db-8cfeaef41bc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:46.194170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6738d-79df-4945-b8db-8cfeaef41bc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:46.204988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6738d-79df-4945-b8db-8cfeaef41bc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:46.206712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6738d-79df-4945-b8db-8cfeaef41bc5 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:46.230366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1049/2105] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:46 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:01:47.771133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4efc7-6d34-432e-a9e9-42d65c455bce None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:47.835818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4efc7-6d34-432e-a9e9-42d65c455bce None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:47.840556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:47.841190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:47.841500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:47.841744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] PATH_INFO: `/v3/users/228e7168799543eb8ab1ac79ed79559f` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:47.842879 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=228e7168799543eb8ab1ac79ed79559f)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:47.844861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:48.061177 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228e7168799543eb8ab1ac79ed79559f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:48.062484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228e7168799543eb8ab1ac79ed79559f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:48.063124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228e7168799543eb8ab1ac79ed79559f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:48.143391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228e7168799543eb8ab1ac79ed79559f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:48.155797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228e7168799543eb8ab1ac79ed79559f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:48.167742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1057/2106] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:47 2022] DELETE /identity/v3/users/228e7168799543eb8ab1ac79ed79559f => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:48.173295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057c914-c5f1-43c0-a73c-6265cc05fe1f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:48.173778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057c914-c5f1-43c0-a73c-6265cc05fe1f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:48.173875 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057c914-c5f1-43c0-a73c-6265cc05fe1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:48.193305 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e057c914-c5f1-43c0-a73c-6265cc05fe1f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:48.194501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e057c914-c5f1-43c0-a73c-6265cc05fe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:48.253780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1050/2107] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:48.263624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:48.265331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:48.318122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:48.336429 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90a9ec-4de3-49b8-897d-e8a1344dfcc8 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:48.340250 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5fd93-eb31-4534-a4d4-21ae6e95a757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:48.340866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5fd93-eb31-4534-a4d4-21ae6e95a757 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:48.341144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5fd93-eb31-4534-a4d4-21ae6e95a757 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:48.341414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d5fd93-eb31-4534-a4d4-21ae6e95a757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:48.353181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5fd93-eb31-4534-a4d4-21ae6e95a757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:48.354957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d5fd93-eb31-4534-a4d4-21ae6e95a757 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:48.367581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1058/2108] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:01:49.002346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057c914-c5f1-43c0-a73c-6265cc05fe1f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:49.003979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e057c914-c5f1-43c0-a73c-6265cc05fe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:49.016372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057c914-c5f1-43c0-a73c-6265cc05fe1f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:49.021297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:49.021937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:49.022254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:49.022527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] PATH_INFO: `/v3/projects/33cab39ee8ff4c1fb904c60e04916fdb` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:49.023651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33cab39ee8ff4c1fb904c60e04916fdb)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:49.025565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:49.148349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:49.151175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33cab39ee8ff4c1fb904c60e04916fdb', 'request_id': 'req-98fafc9d-1a56-4443-82a8-195a6e43a672'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:49.158576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33cab39ee8ff4c1fb904c60e04916fdb', 'request_id': 'req-98fafc9d-1a56-4443-82a8-195a6e43a672'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:49.235273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33cab39ee8ff4c1fb904c60e04916fdb', 'request_id': 'req-98fafc9d-1a56-4443-82a8-195a6e43a672'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:49.248460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] The token cache is being invalidate because project 33cab39ee8ff4c1fb904c60e04916fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:49.248989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:49.250378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:49.252630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1051/2109] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:49 2022] DELETE /identity/v3/projects/33cab39ee8ff4c1fb904c60e04916fdb => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:49.256378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5fd93-eb31-4534-a4d4-21ae6e95a757 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:49.273466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d5fd93-eb31-4534-a4d4-21ae6e95a757 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:49.329376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:49.330123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:49.330518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:49.330993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] PATH_INFO: `/v3/users/ceee9d9a9301460880f0a4bb6f82a5ee` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:49.332200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceee9d9a9301460880f0a4bb6f82a5ee)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:49.334139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:49.551783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceee9d9a9301460880f0a4bb6f82a5ee'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:49.552454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceee9d9a9301460880f0a4bb6f82a5ee'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:49.553091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceee9d9a9301460880f0a4bb6f82a5ee'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:49.701217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceee9d9a9301460880f0a4bb6f82a5ee'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:49.723227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-32d36d22-509d-40dc-b183-1d2330d1d3c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceee9d9a9301460880f0a4bb6f82a5ee'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:49.739549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1059/2110] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:01:49 2022] DELETE /identity/v3/users/ceee9d9a9301460880f0a4bb6f82a5ee => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:50.316688 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:50.318138 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:50.332069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fafc9d-1a56-4443-82a8-195a6e43a672 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:50.388067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:50.389427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:50.390237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:50.390571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] PATH_INFO: `/v3/projects/92e7fd26fa1e4af8ab2febaec4a866d5` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:50.391750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92e7fd26fa1e4af8ab2febaec4a866d5)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:50.393846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:50.540815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92e7fd26fa1e4af8ab2febaec4a866d5', 'request_id': 'req-3ad23e0f-4e21-4463-b8a2-d2e286407a46'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:50.547631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92e7fd26fa1e4af8ab2febaec4a866d5', 'request_id': 'req-3ad23e0f-4e21-4463-b8a2-d2e286407a46'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:50.693433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92e7fd26fa1e4af8ab2febaec4a866d5', 'request_id': 'req-3ad23e0f-4e21-4463-b8a2-d2e286407a46'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:50.707775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] The token cache is being invalidate because project 92e7fd26fa1e4af8ab2febaec4a866d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:01:50.708068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:50.708773 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3ad23e0f-4e21-4463-b8a2-d2e286407a46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:01:50.710509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1052/2111] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:01:50 2022] DELETE /identity/v3/projects/92e7fd26fa1e4af8ab2febaec4a866d5 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:50.715209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0456df-00c3-4d52-a982-1b48d5d18d4d None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:50.715663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0456df-00c3-4d52-a982-1b48d5d18d4d None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:50.715851 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0456df-00c3-4d52-a982-1b48d5d18d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:50.735610 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ff0456df-00c3-4d52-a982-1b48d5d18d4d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:50.736870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ff0456df-00c3-4d52-a982-1b48d5d18d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:50.789609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:50.799112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1060/2112] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:50.806911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f7144-4906-4df2-baf9-ed4bf6080c1e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:50.807201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f7144-4906-4df2-baf9-ed4bf6080c1e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:50.807471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f7144-4906-4df2-baf9-ed4bf6080c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:50.828265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2f5f7144-4906-4df2-baf9-ed4bf6080c1e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:50.829466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2f5f7144-4906-4df2-baf9-ed4bf6080c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:50.837295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1053/2113] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:50.841525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0456df-00c3-4d52-a982-1b48d5d18d4d None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:50.852006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0456df-00c3-4d52-a982-1b48d5d18d4d None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:50.856808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc7416-ac25-4cc6-858d-72457cd229c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:50.857407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc7416-ac25-4cc6-858d-72457cd229c0 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:50.857688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc7416-ac25-4cc6-858d-72457cd229c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:50.857963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc7416-ac25-4cc6-858d-72457cd229c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:50.858646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc7416-ac25-4cc6-858d-72457cd229c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:01:50.859167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc7416-ac25-4cc6-858d-72457cd229c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:50.860531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc7416-ac25-4cc6-858d-72457cd229c0 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:50.872442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1061/2114] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:01:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:01:50.877758 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f7144-4906-4df2-baf9-ed4bf6080c1e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:50.887286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f7144-4906-4df2-baf9-ed4bf6080c1e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:50.893304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ab529-6bb6-4e7d-82e0-daa2b93537b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:50.893919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ab529-6bb6-4e7d-82e0-daa2b93537b2 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:50.898157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ab529-6bb6-4e7d-82e0-daa2b93537b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:50.898157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ab529-6bb6-4e7d-82e0-daa2b93537b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:50.898157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ab529-6bb6-4e7d-82e0-daa2b93537b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:50.898157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ab529-6bb6-4e7d-82e0-daa2b93537b2 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:50.980597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1054/2115] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:50 2022] POST /identity/v3/projects => generated 384 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:50.985225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc7416-ac25-4cc6-858d-72457cd229c0 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:50.995468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc7416-ac25-4cc6-858d-72457cd229c0 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:51.057456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f946d5-47a6-45c5-859f-7936a10dd66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:51.058088 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f946d5-47a6-45c5-859f-7936a10dd66e admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:51.058371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f946d5-47a6-45c5-859f-7936a10dd66e admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:51.058643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f946d5-47a6-45c5-859f-7936a10dd66e admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:51.059321 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f946d5-47a6-45c5-859f-7936a10dd66e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:51.061234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f946d5-47a6-45c5-859f-7936a10dd66e admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:51.174487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1062/2116] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:50 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:51.183217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ab529-6bb6-4e7d-82e0-daa2b93537b2 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:51.194285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ab529-6bb6-4e7d-82e0-daa2b93537b2 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:51.198881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5eefe-1628-4b41-ad62-cac2d4f17946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:51.199511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5eefe-1628-4b41-ad62-cac2d4f17946 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:51.199785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5eefe-1628-4b41-ad62-cac2d4f17946 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:51.200054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5eefe-1628-4b41-ad62-cac2d4f17946 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:51.200691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5eefe-1628-4b41-ad62-cac2d4f17946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:51.202296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5eefe-1628-4b41-ad62-cac2d4f17946 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:51.212331 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1055/2117] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:51 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:51.216518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f946d5-47a6-45c5-859f-7936a10dd66e admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:51.226667 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f946d5-47a6-45c5-859f-7936a10dd66e admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:51.231479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cbba5-ff46-43b3-ac0f-af7c6946c6a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:51.232070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cbba5-ff46-43b3-ac0f-af7c6946c6a0 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:51.232350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cbba5-ff46-43b3-ac0f-af7c6946c6a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:51.232625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cbba5-ff46-43b3-ac0f-af7c6946c6a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:51.233262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cbba5-ff46-43b3-ac0f-af7c6946c6a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:51.235504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cbba5-ff46-43b3-ac0f-af7c6946c6a0 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:51.245798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1063/2118] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:51 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:51.250558 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5eefe-1628-4b41-ad62-cac2d4f17946 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:51.260151 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5eefe-1628-4b41-ad62-cac2d4f17946 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:51.264580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520decfb-5443-437a-9c18-498e5c787e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:51.265713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520decfb-5443-437a-9c18-498e5c787e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:51.265983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520decfb-5443-437a-9c18-498e5c787e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:51.266674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520decfb-5443-437a-9c18-498e5c787e17 admin admin] PATH_INFO: `/v3/projects/5448aa2dd8f449b8b4ceed31dc32250b/users/c11215e65f1c450aa5dd5c482e0f1253/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:51.280959 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520decfb-5443-437a-9c18-498e5c787e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5448aa2dd8f449b8b4ceed31dc32250b, user_id=c11215e65f1c450aa5dd5c482e0f1253, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:51.283609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520decfb-5443-437a-9c18-498e5c787e17 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:51.379396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:51.383920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1056/2119] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:51 2022] PUT /identity/v3/projects/5448aa2dd8f449b8b4ceed31dc32250b/users/c11215e65f1c450aa5dd5c482e0f1253/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:51.397497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925b6aa-9004-4ab4-9a7e-08228ae4aaf5 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:51.397497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925b6aa-9004-4ab4-9a7e-08228ae4aaf5 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:51.397497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925b6aa-9004-4ab4-9a7e-08228ae4aaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:51.424532 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-e925b6aa-9004-4ab4-9a7e-08228ae4aaf5 None None] MFA Rules not processed for user `c11215e65f1c450aa5dd5c482e0f1253`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:51.425666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e925b6aa-9004-4ab4-9a7e-08228ae4aaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:51.490993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1064/2120] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:01:51.495595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520decfb-5443-437a-9c18-498e5c787e17 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:51.556941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520decfb-5443-437a-9c18-498e5c787e17 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:51.562163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2f485-dc67-4f4f-b458-3425fb8dda8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:51.562760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2f485-dc67-4f4f-b458-3425fb8dda8f admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:51.563061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2f485-dc67-4f4f-b458-3425fb8dda8f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:51.563360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2f485-dc67-4f4f-b458-3425fb8dda8f admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:51.564011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2f485-dc67-4f4f-b458-3425fb8dda8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:51.565939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2f485-dc67-4f4f-b458-3425fb8dda8f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:51.642462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1057/2121] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:01:51 2022] POST /identity/v3/projects => generated 386 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:51.647111 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925b6aa-9004-4ab4-9a7e-08228ae4aaf5 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:51.658524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925b6aa-9004-4ab4-9a7e-08228ae4aaf5 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:51.706801 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:51.716056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345ecc9-797b-4569-af3a-d989d18f082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:51.716660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345ecc9-797b-4569-af3a-d989d18f082c admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:51.716934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345ecc9-797b-4569-af3a-d989d18f082c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:51.718117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345ecc9-797b-4569-af3a-d989d18f082c admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:51.718117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345ecc9-797b-4569-af3a-d989d18f082c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:51.720136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345ecc9-797b-4569-af3a-d989d18f082c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:51.850810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1065/2122] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:01:51 2022] POST /identity/v3/users => generated 300 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:01:51.856037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2f485-dc67-4f4f-b458-3425fb8dda8f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:51.866696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2f485-dc67-4f4f-b458-3425fb8dda8f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:51.871517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf3470-46a4-4bb5-ab3d-b273f5095655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:51.872126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf3470-46a4-4bb5-ab3d-b273f5095655 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:51.872405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf3470-46a4-4bb5-ab3d-b273f5095655 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:51.872679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf3470-46a4-4bb5-ab3d-b273f5095655 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:51.873317 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf3470-46a4-4bb5-ab3d-b273f5095655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:51.875176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf3470-46a4-4bb5-ab3d-b273f5095655 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:51.886195 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1058/2123] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:51 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:51.891254 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345ecc9-797b-4569-af3a-d989d18f082c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:51.901427 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345ecc9-797b-4569-af3a-d989d18f082c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:51.906811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b5a39-d32e-4b55-9c4b-9a20dff97b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:51.907521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b5a39-d32e-4b55-9c4b-9a20dff97b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:51.907836 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b5a39-d32e-4b55-9c4b-9a20dff97b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:51.908124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b5a39-d32e-4b55-9c4b-9a20dff97b20 admin admin] PATH_INFO: `/v3/projects/9f9875cf66fa4305b463a2a9ccec119f/users/4a8389caec244dee8de40d28b613a874/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:51.924469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b5a39-d32e-4b55-9c4b-9a20dff97b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9875cf66fa4305b463a2a9ccec119f, user_id=4a8389caec244dee8de40d28b613a874, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:51.927153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b5a39-d32e-4b55-9c4b-9a20dff97b20 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:52.046694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1066/2124] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:51 2022] PUT /identity/v3/projects/9f9875cf66fa4305b463a2a9ccec119f/users/4a8389caec244dee8de40d28b613a874/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:01:52.051082 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf3470-46a4-4bb5-ab3d-b273f5095655 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:52.062672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf3470-46a4-4bb5-ab3d-b273f5095655 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:52.122789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd05b6-177c-4579-af3d-5092a4217a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:52.123445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd05b6-177c-4579-af3d-5092a4217a56 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:52.123715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd05b6-177c-4579-af3d-5092a4217a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:52.124500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd05b6-177c-4579-af3d-5092a4217a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:52.125856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd05b6-177c-4579-af3d-5092a4217a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:52.127476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd05b6-177c-4579-af3d-5092a4217a56 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:52.141161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1059/2125] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:01:52 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:01:52.145487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b5a39-d32e-4b55-9c4b-9a20dff97b20 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:52.157077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b5a39-d32e-4b55-9c4b-9a20dff97b20 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:52.161828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb0bcc-5cbf-48ec-ad40-468d80bb7a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:52.162473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0bcc-5cbf-48ec-ad40-468d80bb7a42 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:52.162748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0bcc-5cbf-48ec-ad40-468d80bb7a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:52.163029 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb0bcc-5cbf-48ec-ad40-468d80bb7a42 admin admin] PATH_INFO: `/v3/projects/9f9875cf66fa4305b463a2a9ccec119f/users/4a8389caec244dee8de40d28b613a874/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:52.164973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb0bcc-5cbf-48ec-ad40-468d80bb7a42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9875cf66fa4305b463a2a9ccec119f, user_id=4a8389caec244dee8de40d28b613a874, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:52.167618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb0bcc-5cbf-48ec-ad40-468d80bb7a42 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:52.181151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1067/2126] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:01:52 2022] PUT /identity/v3/projects/9f9875cf66fa4305b463a2a9ccec119f/users/4a8389caec244dee8de40d28b613a874/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:01:52.193485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcccdac-b7be-4236-a569-063d9b9646b9 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:52.193825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcccdac-b7be-4236-a569-063d9b9646b9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:52.200497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcccdac-b7be-4236-a569-063d9b9646b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:52.228490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-adcccdac-b7be-4236-a569-063d9b9646b9 None None] MFA Rules not processed for user `4a8389caec244dee8de40d28b613a874`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:01:52.228490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-adcccdac-b7be-4236-a569-063d9b9646b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:52.291561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:01:52.302039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1060/2127] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:01:52 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:01:52.313518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb0bcc-5cbf-48ec-ad40-468d80bb7a42 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:01:52.315381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-05cb0bcc-5cbf-48ec-ad40-468d80bb7a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:01:52.394536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb0bcc-5cbf-48ec-ad40-468d80bb7a42 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:01:52.398322 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93255955-9f10-45a5-a80b-a213ce40410d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:01:52.398925 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93255955-9f10-45a5-a80b-a213ce40410d service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:01:52.399232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93255955-9f10-45a5-a80b-a213ce40410d service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:01:52.399519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93255955-9f10-45a5-a80b-a213ce40410d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:01:52.461328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93255955-9f10-45a5-a80b-a213ce40410d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:01:52.462764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93255955-9f10-45a5-a80b-a213ce40410d service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:01:52.472170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1068/2128] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:01:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:02.467720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:03.017556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:03.494982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5779621-6bd3-4ee4-9a5b-11d334f25b27 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:03.496516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5779621-6bd3-4ee4-9a5b-11d334f25b27 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:03.496516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5779621-6bd3-4ee4-9a5b-11d334f25b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:03.515978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-d5779621-6bd3-4ee4-9a5b-11d334f25b27 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:03.517884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d5779621-6bd3-4ee4-9a5b-11d334f25b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:03.574763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1061/2129] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 05:02:03 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:03.621385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93255955-9f10-45a5-a80b-a213ce40410d service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:03.622578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-93255955-9f10-45a5-a80b-a213ce40410d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:03.699008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93255955-9f10-45a5-a80b-a213ce40410d service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:03.704390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6560d-142a-4d01-b310-74db4dd2c728 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:03.704933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6560d-142a-4d01-b310-74db4dd2c728 service swift] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:03.705211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6560d-142a-4d01-b310-74db4dd2c728 service swift] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:03.705483 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6560d-142a-4d01-b310-74db4dd2c728 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:03.713883 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6560d-142a-4d01-b310-74db4dd2c728 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:03.715603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6560d-142a-4d01-b310-74db4dd2c728 service swift] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:03.724484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1069/2130] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:02:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:02:04.328533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5779621-6bd3-4ee4-9a5b-11d334f25b27 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:04.329765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d5779621-6bd3-4ee4-9a5b-11d334f25b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:04.342418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5779621-6bd3-4ee4-9a5b-11d334f25b27 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:04.346297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c503d-c550-4146-bf9f-abbe3310e0d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:04.355412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c503d-c550-4146-bf9f-abbe3310e0d7 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:04.355412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c503d-c550-4146-bf9f-abbe3310e0d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:04.355412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c503d-c550-4146-bf9f-abbe3310e0d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:04.359521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c503d-c550-4146-bf9f-abbe3310e0d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:04.360972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c503d-c550-4146-bf9f-abbe3310e0d7 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:04.370885 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1062/2131] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:04 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:06.146202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6560d-142a-4d01-b310-74db4dd2c728 service swift] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:06.149270 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-32b6560d-142a-4d01-b310-74db4dd2c728 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:06.261038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6560d-142a-4d01-b310-74db4dd2c728 service swift] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:06.267529 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:06.268424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:06.268901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:06.269353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] PATH_INFO: `/v3/users/004fd0d9df6540ffa7054863a5146562` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:06.270937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=004fd0d9df6540ffa7054863a5146562)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:06.273051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:06.475139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '004fd0d9df6540ffa7054863a5146562'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:06.475734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '004fd0d9df6540ffa7054863a5146562'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:06.476469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '004fd0d9df6540ffa7054863a5146562'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:06.560976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '004fd0d9df6540ffa7054863a5146562'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:06.576390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '004fd0d9df6540ffa7054863a5146562'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:06.591234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1070/2132] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:06 2022] DELETE /identity/v3/users/004fd0d9df6540ffa7054863a5146562 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:06.595849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b49e5d-7d07-4d8f-81fd-7e2041165e66 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:06.596413 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b49e5d-7d07-4d8f-81fd-7e2041165e66 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:06.596413 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b49e5d-7d07-4d8f-81fd-7e2041165e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:06.616837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-78b49e5d-7d07-4d8f-81fd-7e2041165e66 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:06.627710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-78b49e5d-7d07-4d8f-81fd-7e2041165e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:06.697349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1063/2133] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:06.709683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:06.726326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:06.726326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:06.796656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b721b278-0832-4b1c-8352-5adcc372bb6d admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:06.800505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6e287-02fc-4c9f-ac49-aa1d71a5be23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:06.801076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e287-02fc-4c9f-ac49-aa1d71a5be23 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:06.801344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e287-02fc-4c9f-ac49-aa1d71a5be23 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:06.801607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6e287-02fc-4c9f-ac49-aa1d71a5be23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:06.810523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6e287-02fc-4c9f-ac49-aa1d71a5be23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:06.811981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6e287-02fc-4c9f-ac49-aa1d71a5be23 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:06.821627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1071/2134] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:07.345655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b49e5d-7d07-4d8f-81fd-7e2041165e66 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:07.347130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-78b49e5d-7d07-4d8f-81fd-7e2041165e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:07.365736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b49e5d-7d07-4d8f-81fd-7e2041165e66 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:07.395358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:07.396509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:07.397093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:07.397512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] PATH_INFO: `/v3/projects/0139022fbb4b410695b52da3b45fabc8` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:07.399051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=0139022fbb4b410695b52da3b45fabc8)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:07.401673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:07.579992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0139022fbb4b410695b52da3b45fabc8', 'request_id': 'req-556f1e40-6b80-43a2-8c5c-00912697c4ee'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:07.589022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0139022fbb4b410695b52da3b45fabc8', 'request_id': 'req-556f1e40-6b80-43a2-8c5c-00912697c4ee'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:07.686074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0139022fbb4b410695b52da3b45fabc8', 'request_id': 'req-556f1e40-6b80-43a2-8c5c-00912697c4ee'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:07.700322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] The token cache is being invalidate because project 0139022fbb4b410695b52da3b45fabc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:07.700627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:07.701297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:07.703521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1064/2135] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:07 2022] DELETE /identity/v3/projects/0139022fbb4b410695b52da3b45fabc8 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:07.708708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6e287-02fc-4c9f-ac49-aa1d71a5be23 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:07.724394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6e287-02fc-4c9f-ac49-aa1d71a5be23 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:07.842378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:07.843835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:07.844137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:07.844411 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] PATH_INFO: `/v3/users/89b2435ed2ca4d0b817af5a8447af389` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:07.845755 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b2435ed2ca4d0b817af5a8447af389)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:07.847766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:08.058628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b2435ed2ca4d0b817af5a8447af389'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:08.060114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b2435ed2ca4d0b817af5a8447af389'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:08.061673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b2435ed2ca4d0b817af5a8447af389'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:08.137081 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b2435ed2ca4d0b817af5a8447af389'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:08.152246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-051b9e85-a204-4af2-8835-a45bd3958f02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b2435ed2ca4d0b817af5a8447af389'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:08.168747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1072/2136] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:07 2022] DELETE /identity/v3/users/89b2435ed2ca4d0b817af5a8447af389 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:08.792181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:08.795781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:08.816567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f1e40-6b80-43a2-8c5c-00912697c4ee admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:08.839981 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:08.943269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:08.944574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:08.945090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:08.945785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] PATH_INFO: `/v3/projects/5091062794a94b248c26bbf81d987e30` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:08.947327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5091062794a94b248c26bbf81d987e30)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:08.949864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:09.071794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5091062794a94b248c26bbf81d987e30', 'request_id': 'req-97ee027d-729f-4987-b08b-293948510bf8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:09.079098 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5091062794a94b248c26bbf81d987e30', 'request_id': 'req-97ee027d-729f-4987-b08b-293948510bf8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:09.143340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5091062794a94b248c26bbf81d987e30', 'request_id': 'req-97ee027d-729f-4987-b08b-293948510bf8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:09.159485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] The token cache is being invalidate because project 5091062794a94b248c26bbf81d987e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:09.159797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:09.160473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-97ee027d-729f-4987-b08b-293948510bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:09.162305 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1065/2137] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:08 2022] DELETE /identity/v3/projects/5091062794a94b248c26bbf81d987e30 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:09.166967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd656438-8604-4002-8975-940d6ee68505 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:09.173928 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd656438-8604-4002-8975-940d6ee68505 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:09.173928 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd656438-8604-4002-8975-940d6ee68505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:09.196273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-bd656438-8604-4002-8975-940d6ee68505 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:09.196273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-bd656438-8604-4002-8975-940d6ee68505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:09.229966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:09.281640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1073/2138] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:09.294187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845ed42-aa37-4b6f-9f1e-6e092f60cb1b None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:09.294187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845ed42-aa37-4b6f-9f1e-6e092f60cb1b None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:09.294187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4845ed42-aa37-4b6f-9f1e-6e092f60cb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:09.320987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-4845ed42-aa37-4b6f-9f1e-6e092f60cb1b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:09.322295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4845ed42-aa37-4b6f-9f1e-6e092f60cb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:09.341562 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1066/2139] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:09.345400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd656438-8604-4002-8975-940d6ee68505 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:09.356502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd656438-8604-4002-8975-940d6ee68505 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:09.363027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc09a9-f383-48d0-ba85-bf6efaae743b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:09.365396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc09a9-f383-48d0-ba85-bf6efaae743b admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:09.365396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc09a9-f383-48d0-ba85-bf6efaae743b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:09.365396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc09a9-f383-48d0-ba85-bf6efaae743b admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:09.365396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc09a9-f383-48d0-ba85-bf6efaae743b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:02:09.365396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc09a9-f383-48d0-ba85-bf6efaae743b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:09.366729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc09a9-f383-48d0-ba85-bf6efaae743b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:09.382713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1074/2140] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:02:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:02:09.384292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845ed42-aa37-4b6f-9f1e-6e092f60cb1b None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:09.396288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4845ed42-aa37-4b6f-9f1e-6e092f60cb1b None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:09.401971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04d689-a4bc-4374-8f93-6dfd4a415bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:09.402701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04d689-a4bc-4374-8f93-6dfd4a415bef admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:09.403044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04d689-a4bc-4374-8f93-6dfd4a415bef admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:09.403403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04d689-a4bc-4374-8f93-6dfd4a415bef admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:09.404026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04d689-a4bc-4374-8f93-6dfd4a415bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:09.406912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04d689-a4bc-4374-8f93-6dfd4a415bef admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:09.505182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1067/2141] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:02:09 2022] POST /identity/v3/projects => generated 366 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:02:09.510004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc09a9-f383-48d0-ba85-bf6efaae743b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:09.521774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc09a9-f383-48d0-ba85-bf6efaae743b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:09.611473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de49be-0969-484e-897c-44e9c585fa0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:09.615229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de49be-0969-484e-897c-44e9c585fa0f admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:09.615524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de49be-0969-484e-897c-44e9c585fa0f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:09.615804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de49be-0969-484e-897c-44e9c585fa0f admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:09.616595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de49be-0969-484e-897c-44e9c585fa0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:09.618890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de49be-0969-484e-897c-44e9c585fa0f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:09.728203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1075/2142] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:02:09 2022] POST /identity/v3/users => generated 284 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:02:09.735665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04d689-a4bc-4374-8f93-6dfd4a415bef admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:09.753020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04d689-a4bc-4374-8f93-6dfd4a415bef admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:09.761403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d4d2d-6535-4e10-abdf-8fecc056f244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:09.762546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d4d2d-6535-4e10-abdf-8fecc056f244 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:09.762983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d4d2d-6535-4e10-abdf-8fecc056f244 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:09.765250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d4d2d-6535-4e10-abdf-8fecc056f244 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:09.766643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d4d2d-6535-4e10-abdf-8fecc056f244 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:09.778519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d4d2d-6535-4e10-abdf-8fecc056f244 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:09.788307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1068/2143] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:09 2022] GET /identity/v3/roles => generated 1427 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:02:09.790501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de49be-0969-484e-897c-44e9c585fa0f admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:09.804173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de49be-0969-484e-897c-44e9c585fa0f admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:09.818936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d886a2a9-28fd-4745-8bfe-670bee877ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:09.819726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886a2a9-28fd-4745-8bfe-670bee877ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:09.820014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886a2a9-28fd-4745-8bfe-670bee877ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:09.820283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d886a2a9-28fd-4745-8bfe-670bee877ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:09.820937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886a2a9-28fd-4745-8bfe-670bee877ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:09.822983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d886a2a9-28fd-4745-8bfe-670bee877ef5 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:09.845200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1076/2144] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:09 2022] GET /identity/v3/roles => generated 1427 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:02:09.848620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d4d2d-6535-4e10-abdf-8fecc056f244 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:09.887497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d4d2d-6535-4e10-abdf-8fecc056f244 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:09.905569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4870fd-1fef-4468-be0e-a96aebe2c265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:09.912321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4870fd-1fef-4468-be0e-a96aebe2c265 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:09.913684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4870fd-1fef-4468-be0e-a96aebe2c265 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:09.913970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4870fd-1fef-4468-be0e-a96aebe2c265 admin admin] PATH_INFO: `/v3/projects/80394175293a4901862b2cfacc09b18d/users/507f47c5411b45a4bf0c9e565da73d75/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:09.954122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4870fd-1fef-4468-be0e-a96aebe2c265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80394175293a4901862b2cfacc09b18d, user_id=507f47c5411b45a4bf0c9e565da73d75, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:09.957659 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4870fd-1fef-4468-be0e-a96aebe2c265 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:10.047619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1069/2145] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:02:09 2022] PUT /identity/v3/projects/80394175293a4901862b2cfacc09b18d/users/507f47c5411b45a4bf0c9e565da73d75/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:02:10.061149 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69975062-b20d-4d5a-aa1c-acab3388b561 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:10.062044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69975062-b20d-4d5a-aa1c-acab3388b561 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:10.062694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69975062-b20d-4d5a-aa1c-acab3388b561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:10.130122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-69975062-b20d-4d5a-aa1c-acab3388b561 None None] MFA Rules not processed for user `507f47c5411b45a4bf0c9e565da73d75`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:10.139349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-69975062-b20d-4d5a-aa1c-acab3388b561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:10.241253 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1077/2146] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:10.241956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4870fd-1fef-4468-be0e-a96aebe2c265 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:10.307291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:10.345223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4870fd-1fef-4468-be0e-a96aebe2c265 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:10.355057 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc278a-b091-48c1-844a-cbc39608afb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:10.355823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc278a-b091-48c1-844a-cbc39608afb1 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:10.356173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc278a-b091-48c1-844a-cbc39608afb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:10.356517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc278a-b091-48c1-844a-cbc39608afb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:10.359083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc278a-b091-48c1-844a-cbc39608afb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:10.383350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc278a-b091-48c1-844a-cbc39608afb1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:10.542499 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1070/2147] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:02:10 2022] POST /identity/v3/projects => generated 366 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:02:10.547958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69975062-b20d-4d5a-aa1c-acab3388b561 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:10.566596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69975062-b20d-4d5a-aa1c-acab3388b561 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:10.593677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:10.627991 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c275814-7365-48ab-8371-9233322c5c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:10.628509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c275814-7365-48ab-8371-9233322c5c57 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:10.628810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c275814-7365-48ab-8371-9233322c5c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:10.629041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c275814-7365-48ab-8371-9233322c5c57 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:10.629669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c275814-7365-48ab-8371-9233322c5c57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:10.631648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c275814-7365-48ab-8371-9233322c5c57 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:10.769740 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1078/2148] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:02:10 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:02:10.773351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc278a-b091-48c1-844a-cbc39608afb1 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:10.788279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc278a-b091-48c1-844a-cbc39608afb1 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:10.793611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c384aca-f510-43e6-9a42-b91e5da5a7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:10.794366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c384aca-f510-43e6-9a42-b91e5da5a7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:10.794736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c384aca-f510-43e6-9a42-b91e5da5a7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:10.795167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c384aca-f510-43e6-9a42-b91e5da5a7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:10.796337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c384aca-f510-43e6-9a42-b91e5da5a7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:10.798561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c384aca-f510-43e6-9a42-b91e5da5a7b2 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:10.816163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1071/2149] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:10 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:02:10.823049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c275814-7365-48ab-8371-9233322c5c57 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:10.831720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c275814-7365-48ab-8371-9233322c5c57 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:10.836717 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5a712-bee0-4c78-ac28-c70e52a69ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:10.837504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5a712-bee0-4c78-ac28-c70e52a69ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:10.837909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5a712-bee0-4c78-ac28-c70e52a69ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:10.838273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5a712-bee0-4c78-ac28-c70e52a69ef7 admin admin] PATH_INFO: `/v3/projects/229610c5441448e68ce3a3ed443727e2/users/49961f8a893145fe872093de6d232450/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:10.856989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5a712-bee0-4c78-ac28-c70e52a69ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=229610c5441448e68ce3a3ed443727e2, user_id=49961f8a893145fe872093de6d232450, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:10.860531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5a712-bee0-4c78-ac28-c70e52a69ef7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:10.940927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1079/2150] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:02:10 2022] PUT /identity/v3/projects/229610c5441448e68ce3a3ed443727e2/users/49961f8a893145fe872093de6d232450/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:10.945998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c384aca-f510-43e6-9a42-b91e5da5a7b2 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:10.964438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c384aca-f510-43e6-9a42-b91e5da5a7b2 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:11.055635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558270c0-701a-49a9-b302-4b68731d240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:11.056820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558270c0-701a-49a9-b302-4b68731d240e admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:11.057175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558270c0-701a-49a9-b302-4b68731d240e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:11.057658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558270c0-701a-49a9-b302-4b68731d240e admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:11.058549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558270c0-701a-49a9-b302-4b68731d240e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:11.060427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558270c0-701a-49a9-b302-4b68731d240e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:11.079798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1072/2151] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:10 2022] GET /identity/v3/roles => generated 1427 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:02:11.080436 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5a712-bee0-4c78-ac28-c70e52a69ef7 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:11.093367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5a712-bee0-4c78-ac28-c70e52a69ef7 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:11.099479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce00dcc-1b48-45c5-8ba5-4732059b813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:11.100142 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce00dcc-1b48-45c5-8ba5-4732059b813d admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:11.100423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce00dcc-1b48-45c5-8ba5-4732059b813d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:11.100690 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce00dcc-1b48-45c5-8ba5-4732059b813d admin admin] PATH_INFO: `/v3/projects/229610c5441448e68ce3a3ed443727e2/users/49961f8a893145fe872093de6d232450/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:11.102726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce00dcc-1b48-45c5-8ba5-4732059b813d admin admin] RBAC: Authorizing `identity:create_grant(project_id=229610c5441448e68ce3a3ed443727e2, user_id=49961f8a893145fe872093de6d232450, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:11.105694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce00dcc-1b48-45c5-8ba5-4732059b813d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:11.121349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1080/2152] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:02:11 2022] PUT /identity/v3/projects/229610c5441448e68ce3a3ed443727e2/users/49961f8a893145fe872093de6d232450/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:02:11.131889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee869b07-747c-4325-ba69-2964a444c3a9 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:11.132761 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee869b07-747c-4325-ba69-2964a444c3a9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:11.132761 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee869b07-747c-4325-ba69-2964a444c3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:11.169973 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ee869b07-747c-4325-ba69-2964a444c3a9 None None] MFA Rules not processed for user `49961f8a893145fe872093de6d232450`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:11.176654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ee869b07-747c-4325-ba69-2964a444c3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:11.271072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1073/2153] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:11.294195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce00dcc-1b48-45c5-8ba5-4732059b813d admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:11.298710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cce00dcc-1b48-45c5-8ba5-4732059b813d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:11.381469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce00dcc-1b48-45c5-8ba5-4732059b813d admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:11.385986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512f5a5-c7c6-4621-a005-85b3cadff910 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:11.386659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512f5a5-c7c6-4621-a005-85b3cadff910 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:11.387056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512f5a5-c7c6-4621-a005-85b3cadff910 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:11.387388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a512f5a5-c7c6-4621-a005-85b3cadff910 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:11.406477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512f5a5-c7c6-4621-a005-85b3cadff910 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:11.408142 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a512f5a5-c7c6-4621-a005-85b3cadff910 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:11.440776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1081/2154] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:11.650160 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee869b07-747c-4325-ba69-2964a444c3a9 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:11.667307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee869b07-747c-4325-ba69-2964a444c3a9 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:11.676122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e21fa4-5ccc-488c-9321-5d635e4c7b06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:11.677041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21fa4-5ccc-488c-9321-5d635e4c7b06 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:11.677327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21fa4-5ccc-488c-9321-5d635e4c7b06 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:11.677774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e21fa4-5ccc-488c-9321-5d635e4c7b06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:11.727409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:11.772291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e21fa4-5ccc-488c-9321-5d635e4c7b06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:11.773790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e21fa4-5ccc-488c-9321-5d635e4c7b06 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:11.786201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1074/2155] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:11.790854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512f5a5-c7c6-4621-a005-85b3cadff910 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:11.794906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a512f5a5-c7c6-4621-a005-85b3cadff910 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:11.910525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a512f5a5-c7c6-4621-a005-85b3cadff910 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:11.916251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:11.917228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:11.917705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:11.918080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] PATH_INFO: `/v3/users/c11215e65f1c450aa5dd5c482e0f1253` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:11.919564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c11215e65f1c450aa5dd5c482e0f1253)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:11.922687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:11.965641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:12.182940 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c11215e65f1c450aa5dd5c482e0f1253'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:12.183837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c11215e65f1c450aa5dd5c482e0f1253'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:12.184434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c11215e65f1c450aa5dd5c482e0f1253'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:12.282196 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c11215e65f1c450aa5dd5c482e0f1253'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:12.307304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c11215e65f1c450aa5dd5c482e0f1253'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:12.323698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1082/2156] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:11 2022] DELETE /identity/v3/users/c11215e65f1c450aa5dd5c482e0f1253 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:12.330076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5083d8c-6ff9-4bb3-9c3a-6d07b7cbb17f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:12.332559 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5083d8c-6ff9-4bb3-9c3a-6d07b7cbb17f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:12.332559 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5083d8c-6ff9-4bb3-9c3a-6d07b7cbb17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:12.354842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-f5083d8c-6ff9-4bb3-9c3a-6d07b7cbb17f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:12.358863 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f5083d8c-6ff9-4bb3-9c3a-6d07b7cbb17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:12.447244 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1075/2157] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:12.458977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:12.460524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:12.587652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12b196-ec47-4cae-87bd-ad1d2b3545a3 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:12.587943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12b196-ec47-4cae-87bd-ad1d2b3545a3 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:12.588224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe12b196-ec47-4cae-87bd-ad1d2b3545a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:12.617471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-fe12b196-ec47-4cae-87bd-ad1d2b3545a3 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:12.618716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-fe12b196-ec47-4cae-87bd-ad1d2b3545a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:12.623154 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f502a15-15c9-4f04-bb91-560a585a296c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:12.632534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d60ca-2b4f-47de-b7a1-8fc2565e67d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:12.633260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d60ca-2b4f-47de-b7a1-8fc2565e67d6 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:12.633655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d60ca-2b4f-47de-b7a1-8fc2565e67d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:12.634018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d60ca-2b4f-47de-b7a1-8fc2565e67d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:12.654808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d60ca-2b4f-47de-b7a1-8fc2565e67d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:12.657887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d60ca-2b4f-47de-b7a1-8fc2565e67d6 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:12.670610 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1083/2158] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:12.700775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1076/2159] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 05:02:12 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:12.714932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d60ca-2b4f-47de-b7a1-8fc2565e67d6 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:12.716471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2c2d60ca-2b4f-47de-b7a1-8fc2565e67d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:12.846830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d60ca-2b4f-47de-b7a1-8fc2565e67d6 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:12.854644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333039e0-73db-4fbb-a4ca-84ee7cba0647 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:12.855562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333039e0-73db-4fbb-a4ca-84ee7cba0647 service swift] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:12.856113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333039e0-73db-4fbb-a4ca-84ee7cba0647 service swift] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:12.856587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333039e0-73db-4fbb-a4ca-84ee7cba0647 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:12.870802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333039e0-73db-4fbb-a4ca-84ee7cba0647 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:12.872576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333039e0-73db-4fbb-a4ca-84ee7cba0647 service swift] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:12.882249 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1084/2160] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:02:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:02:13.268414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12b196-ec47-4cae-87bd-ad1d2b3545a3 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:13.270002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-fe12b196-ec47-4cae-87bd-ad1d2b3545a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:13.289562 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe12b196-ec47-4cae-87bd-ad1d2b3545a3 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:13.295163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:13.297573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:13.297573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:13.297862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] PATH_INFO: `/v3/projects/5448aa2dd8f449b8b4ceed31dc32250b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:13.299976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5448aa2dd8f449b8b4ceed31dc32250b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:13.302046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:13.460951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5448aa2dd8f449b8b4ceed31dc32250b', 'request_id': 'req-7fc89472-273c-4861-aac9-78cdd8116c91'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:13.480083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5448aa2dd8f449b8b4ceed31dc32250b', 'request_id': 'req-7fc89472-273c-4861-aac9-78cdd8116c91'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:13.552211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5448aa2dd8f449b8b4ceed31dc32250b', 'request_id': 'req-7fc89472-273c-4861-aac9-78cdd8116c91'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:13.566252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] The token cache is being invalidate because project 5448aa2dd8f449b8b4ceed31dc32250b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:13.566824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:13.567828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7fc89472-273c-4861-aac9-78cdd8116c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:13.571687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1077/2161] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:13 2022] DELETE /identity/v3/projects/5448aa2dd8f449b8b4ceed31dc32250b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:13.575992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333039e0-73db-4fbb-a4ca-84ee7cba0647 service swift] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:13.589805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333039e0-73db-4fbb-a4ca-84ee7cba0647 service swift] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:13.636531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:13.685731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:13.686807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:13.687207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:13.687831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] PATH_INFO: `/v3/users/4a8389caec244dee8de40d28b613a874` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:13.690221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a8389caec244dee8de40d28b613a874)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:13.692694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:13.930198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a8389caec244dee8de40d28b613a874'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:13.931924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a8389caec244dee8de40d28b613a874'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:13.937332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a8389caec244dee8de40d28b613a874'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:13.985477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a8389caec244dee8de40d28b613a874'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:14.019706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-544b46fc-cbec-41b7-b4e0-8f11954476a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a8389caec244dee8de40d28b613a874'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:14.041685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1085/2162] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:13 2022] DELETE /identity/v3/users/4a8389caec244dee8de40d28b613a874 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:14.324150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ccabf-7a27-42fa-b950-401dba37ad2a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:14.324620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ccabf-7a27-42fa-b950-401dba37ad2a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:14.325388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ccabf-7a27-42fa-b950-401dba37ad2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:14.369292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:14.390821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b533-abea-4e2a-8b98-d92011b792e7 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:14.391056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b533-abea-4e2a-8b98-d92011b792e7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:14.391347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0063b533-abea-4e2a-8b98-d92011b792e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:14.398904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-0a8ccabf-7a27-42fa-b950-401dba37ad2a None None] MFA Rules not processed for user `51926dd0d6934aed907806a5cb14dc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:14.400998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0a8ccabf-7a27-42fa-b950-401dba37ad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:14.419680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-0063b533-abea-4e2a-8b98-d92011b792e7 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:14.421403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0063b533-abea-4e2a-8b98-d92011b792e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:14.504512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1086/2163] 10.222.0.105 () {58 vars in 972 bytes} [Wed Aug 24 05:02:14 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:02:14.544930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063b533-abea-4e2a-8b98-d92011b792e7 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:14.546487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0063b533-abea-4e2a-8b98-d92011b792e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:14.552847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1078/2164] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:14 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:14.628173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:14.644928 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0063b533-abea-4e2a-8b98-d92011b792e7 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:14.650760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc74609f-f10b-45e0-b16c-422cfa2e597f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:14.651691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74609f-f10b-45e0-b16c-422cfa2e597f service swift] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:14.652052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74609f-f10b-45e0-b16c-422cfa2e597f service swift] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:14.652328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc74609f-f10b-45e0-b16c-422cfa2e597f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:14.663919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74609f-f10b-45e0-b16c-422cfa2e597f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:14.668553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc74609f-f10b-45e0-b16c-422cfa2e597f service swift] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:14.678290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1087/2165] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:02:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:02:14.701654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ccabf-7a27-42fa-b950-401dba37ad2a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:14.704556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0a8ccabf-7a27-42fa-b950-401dba37ad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:14.884433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ccabf-7a27-42fa-b950-401dba37ad2a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:14.890408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:14.891379 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:14.891873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:14.892341 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] PATH_INFO: `/v3/projects/9f9875cf66fa4305b463a2a9ccec119f` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:14.896047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9875cf66fa4305b463a2a9ccec119f)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:14.898351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:15.045161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9875cf66fa4305b463a2a9ccec119f', 'request_id': 'req-b0c44f5e-3534-47de-88e2-8622dc323a7e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:15.057740 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9875cf66fa4305b463a2a9ccec119f', 'request_id': 'req-b0c44f5e-3534-47de-88e2-8622dc323a7e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:15.109943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9875cf66fa4305b463a2a9ccec119f', 'request_id': 'req-b0c44f5e-3534-47de-88e2-8622dc323a7e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:15.125113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] The token cache is being invalidate because project 9f9875cf66fa4305b463a2a9ccec119f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:15.125406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:15.126071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b0c44f5e-3534-47de-88e2-8622dc323a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:15.127721 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1079/2166] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:14 2022] DELETE /identity/v3/projects/9f9875cf66fa4305b463a2a9ccec119f => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:15.133919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1417b-7c51-45c1-81dc-cd4f149c9842 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.134205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1417b-7c51-45c1-81dc-cd4f149c9842 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.138625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1417b-7c51-45c1-81dc-cd4f149c9842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.156022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-4ef1417b-7c51-45c1-81dc-cd4f149c9842 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:15.159887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4ef1417b-7c51-45c1-81dc-cd4f149c9842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:15.236244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1088/2167] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:15.244882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c68db-2daf-4c10-9966-96785ea8632a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.248980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c68db-2daf-4c10-9966-96785ea8632a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.248980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c68db-2daf-4c10-9966-96785ea8632a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.269854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9f7c68db-2daf-4c10-9966-96785ea8632a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:15.271295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9f7c68db-2daf-4c10-9966-96785ea8632a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:15.283898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1080/2168] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:15.288107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1417b-7c51-45c1-81dc-cd4f149c9842 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:15.300702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1417b-7c51-45c1-81dc-cd4f149c9842 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:15.323350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81189650-b465-4099-a701-3c276e3643c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:15.324906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81189650-b465-4099-a701-3c276e3643c6 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.325341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81189650-b465-4099-a701-3c276e3643c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.325769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81189650-b465-4099-a701-3c276e3643c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.328479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81189650-b465-4099-a701-3c276e3643c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:02:15.328935 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81189650-b465-4099-a701-3c276e3643c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:15.333139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81189650-b465-4099-a701-3c276e3643c6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:15.353471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1089/2169] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:02:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:02:15.357810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c68db-2daf-4c10-9966-96785ea8632a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:15.368632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c68db-2daf-4c10-9966-96785ea8632a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:15.377018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd24e7a-fe72-4327-b781-943b72608e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:15.377914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd24e7a-fe72-4327-b781-943b72608e3d admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.378196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd24e7a-fe72-4327-b781-943b72608e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.378626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd24e7a-fe72-4327-b781-943b72608e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.379482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd24e7a-fe72-4327-b781-943b72608e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:15.381557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd24e7a-fe72-4327-b781-943b72608e3d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:15.508264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1081/2170] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:02:15 2022] POST /identity/v3/projects => generated 382 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:02:15.512066 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81189650-b465-4099-a701-3c276e3643c6 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:15.525337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81189650-b465-4099-a701-3c276e3643c6 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:15.625102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b75814-ea46-464c-be8a-e911dcd3b974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:15.625958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b75814-ea46-464c-be8a-e911dcd3b974 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.626069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b75814-ea46-464c-be8a-e911dcd3b974 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.626296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b75814-ea46-464c-be8a-e911dcd3b974 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.626956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b75814-ea46-464c-be8a-e911dcd3b974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:15.629029 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b75814-ea46-464c-be8a-e911dcd3b974 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:15.660675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba77a6-0c1d-4713-ad6c-4f6d9573b871 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.661185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba77a6-0c1d-4713-ad6c-4f6d9573b871 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.661264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba77a6-0c1d-4713-ad6c-4f6d9573b871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.682598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-f8ba77a6-0c1d-4713-ad6c-4f6d9573b871 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:15.683892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f8ba77a6-0c1d-4713-ad6c-4f6d9573b871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:15.691731 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:15.714480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1090/2171] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:02:15 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:02:15.718214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b75814-ea46-464c-be8a-e911dcd3b974 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:15.737399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b75814-ea46-464c-be8a-e911dcd3b974 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:15.745647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea53c3-35d2-48ff-bbd9-1922ddfaedcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:15.747397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea53c3-35d2-48ff-bbd9-1922ddfaedcd admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.747781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea53c3-35d2-48ff-bbd9-1922ddfaedcd admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.748157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea53c3-35d2-48ff-bbd9-1922ddfaedcd admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.749815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea53c3-35d2-48ff-bbd9-1922ddfaedcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:15.751892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea53c3-35d2-48ff-bbd9-1922ddfaedcd admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:15.770746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1091/2172] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:15 2022] GET /identity/v3/roles => generated 1427 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:02:15.774598 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea53c3-35d2-48ff-bbd9-1922ddfaedcd admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:15.781790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1082/2173] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 05:02:15 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:15.787360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea53c3-35d2-48ff-bbd9-1922ddfaedcd admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:15.794996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba77a6-0c1d-4713-ad6c-4f6d9573b871 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:15.797489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f8ba77a6-0c1d-4713-ad6c-4f6d9573b871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:15.799312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a34724-c381-44f7-b237-752d8ff2fb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:15.800503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a34724-c381-44f7-b237-752d8ff2fb57 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.801904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a34724-c381-44f7-b237-752d8ff2fb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.802803 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a34724-c381-44f7-b237-752d8ff2fb57 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.803598 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a34724-c381-44f7-b237-752d8ff2fb57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:15.808212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a34724-c381-44f7-b237-752d8ff2fb57 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:15.826325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1092/2174] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:15 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:02:15.830800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a34724-c381-44f7-b237-752d8ff2fb57 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:15.841769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a34724-c381-44f7-b237-752d8ff2fb57 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:15.848283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a89dc-8064-4fa9-a682-0563605f9d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:15.849107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a89dc-8064-4fa9-a682-0563605f9d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.849596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a89dc-8064-4fa9-a682-0563605f9d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.849973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a89dc-8064-4fa9-a682-0563605f9d7d admin admin] PATH_INFO: `/v3/projects/2dd4d1addbd348c0a9608c7b3fe27157/users/3cc9d727a1e044a397b8b8b9883d5dcb/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.876771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a89dc-8064-4fa9-a682-0563605f9d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd4d1addbd348c0a9608c7b3fe27157, user_id=3cc9d727a1e044a397b8b8b9883d5dcb, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:15.880724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a89dc-8064-4fa9-a682-0563605f9d7d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:15.909130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba77a6-0c1d-4713-ad6c-4f6d9573b871 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:15.915053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef495a-d6eb-4222-9cc5-d9202dde6bbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:15.915370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef495a-d6eb-4222-9cc5-d9202dde6bbf service swift] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.915790 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef495a-d6eb-4222-9cc5-d9202dde6bbf service swift] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.916222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef495a-d6eb-4222-9cc5-d9202dde6bbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.937235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef495a-d6eb-4222-9cc5-d9202dde6bbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:15.940761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1093/2175] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:02:15 2022] PUT /identity/v3/projects/2dd4d1addbd348c0a9608c7b3fe27157/users/3cc9d727a1e044a397b8b8b9883d5dcb/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:15.942239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef495a-d6eb-4222-9cc5-d9202dde6bbf service swift] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:15.945513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ea34-4cc0-488f-8b18-1dc6026a6213 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:15.945802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ea34-4cc0-488f-8b18-1dc6026a6213 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:15.946274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ea34-4cc0-488f-8b18-1dc6026a6213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:15.959528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1083/2176] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:02:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:02:15.981309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-1b40ea34-4cc0-488f-8b18-1dc6026a6213 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:15.982577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1b40ea34-4cc0-488f-8b18-1dc6026a6213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:15.995612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:16.059914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1094/2177] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:16.069789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef495a-d6eb-4222-9cc5-d9202dde6bbf service swift] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:16.071360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9cef495a-d6eb-4222-9cc5-d9202dde6bbf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:16.181730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef495a-d6eb-4222-9cc5-d9202dde6bbf service swift] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:16.259204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3965d-449d-4e48-b43d-8add798edaee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:16.259811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3965d-449d-4e48-b43d-8add798edaee service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:16.260096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3965d-449d-4e48-b43d-8add798edaee service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:16.260372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3965d-449d-4e48-b43d-8add798edaee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:16.271426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3965d-449d-4e48-b43d-8add798edaee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:16.273121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3965d-449d-4e48-b43d-8add798edaee service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:16.287713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1084/2178] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:16.653680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bb035-3c29-456c-9831-38a51d850c0e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:16.654148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bb035-3c29-456c-9831-38a51d850c0e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:16.654264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0bb035-3c29-456c-9831-38a51d850c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:16.679940 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-7b0bb035-3c29-456c-9831-38a51d850c0e None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:16.681698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-7b0bb035-3c29-456c-9831-38a51d850c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:16.754020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1095/2179] 10.222.0.105 () {58 vars in 972 bytes} [Wed Aug 24 05:02:16 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:16.769696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3965d-449d-4e48-b43d-8add798edaee service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:16.776538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:16.886334 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3965d-449d-4e48-b43d-8add798edaee service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:16.910998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09474495-0cb3-4b5a-801b-1c0a7188e349 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:16.911658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09474495-0cb3-4b5a-801b-1c0a7188e349 service swift] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:16.911915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09474495-0cb3-4b5a-801b-1c0a7188e349 service swift] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:16.912177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09474495-0cb3-4b5a-801b-1c0a7188e349 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:16.931193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09474495-0cb3-4b5a-801b-1c0a7188e349 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:16.932763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09474495-0cb3-4b5a-801b-1c0a7188e349 service swift] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:16.954788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1085/2180] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:02:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:02:18.102856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9f585-efe0-444e-9f16-ebca483a1933 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:18.104075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9f585-efe0-444e-9f16-ebca483a1933 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:18.104356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9f585-efe0-444e-9f16-ebca483a1933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:18.205834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2be9f585-efe0-444e-9f16-ebca483a1933 None None] MFA Rules not processed for user `3cc9d727a1e044a397b8b8b9883d5dcb`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:18.207403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2be9f585-efe0-444e-9f16-ebca483a1933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:18.294924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1096/2181] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:18.399707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09474495-0cb3-4b5a-801b-1c0a7188e349 service swift] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:18.404067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-09474495-0cb3-4b5a-801b-1c0a7188e349 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:18.492872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09474495-0cb3-4b5a-801b-1c0a7188e349 service swift] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:18.498609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b98f8d-bde0-4940-a183-070f560c28cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:18.504070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b98f8d-bde0-4940-a183-070f560c28cd service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:18.504070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b98f8d-bde0-4940-a183-070f560c28cd service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:18.504070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b98f8d-bde0-4940-a183-070f560c28cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:18.518147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b98f8d-bde0-4940-a183-070f560c28cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:18.519945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b98f8d-bde0-4940-a183-070f560c28cd service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:18.546597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1086/2182] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:20.596568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9f585-efe0-444e-9f16-ebca483a1933 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:20.598087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2be9f585-efe0-444e-9f16-ebca483a1933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:20.627166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9f585-efe0-444e-9f16-ebca483a1933 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:20.632760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:20.633748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:20.634293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:20.634741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] PATH_INFO: `/v3/users/507f47c5411b45a4bf0c9e565da73d75` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:20.636639 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] RBAC: Authorizing `identity:delete_user(user_id=507f47c5411b45a4bf0c9e565da73d75)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:20.639189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:20.795130 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '507f47c5411b45a4bf0c9e565da73d75'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:20.799390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '507f47c5411b45a4bf0c9e565da73d75'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:20.799565 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '507f47c5411b45a4bf0c9e565da73d75'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:20.802815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:20.881743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '507f47c5411b45a4bf0c9e565da73d75'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:20.909684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '507f47c5411b45a4bf0c9e565da73d75'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:20.941028 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1097/2183] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:20 2022] DELETE /identity/v3/users/507f47c5411b45a4bf0c9e565da73d75 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:20.943103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566378-ad5a-460f-b209-188fa2b304b2 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:20.943583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566378-ad5a-460f-b209-188fa2b304b2 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:20.943836 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3566378-ad5a-460f-b209-188fa2b304b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:20.989211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-d3566378-ad5a-460f-b209-188fa2b304b2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:20.993307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d3566378-ad5a-460f-b209-188fa2b304b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:21.114331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:21.116112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:21.217448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1087/2184] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:21.230459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566378-ad5a-460f-b209-188fa2b304b2 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:21.230459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d3566378-ad5a-460f-b209-188fa2b304b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:21.245690 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b0398-35ac-40f9-90ac-6c171e306128 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:21.251837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:21.252695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:21.253127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:21.258098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] PATH_INFO: `/v3/users/ebea1b500dd44a9888d5cd18c55b0909` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:21.258098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebea1b500dd44a9888d5cd18c55b0909)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:21.258098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:21.305130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:21.330347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3566378-ad5a-460f-b209-188fa2b304b2 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:21.335463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e710e9-5729-48f6-9ef2-64c5a16d44ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:21.341488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e710e9-5729-48f6-9ef2-64c5a16d44ed service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:21.341488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e710e9-5729-48f6-9ef2-64c5a16d44ed service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:21.341488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e710e9-5729-48f6-9ef2-64c5a16d44ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:21.349881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e710e9-5729-48f6-9ef2-64c5a16d44ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:21.351584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e710e9-5729-48f6-9ef2-64c5a16d44ed service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:21.365607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1088/2185] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:21.475191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebea1b500dd44a9888d5cd18c55b0909'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:21.479261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebea1b500dd44a9888d5cd18c55b0909'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:21.480001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebea1b500dd44a9888d5cd18c55b0909'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:21.560202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebea1b500dd44a9888d5cd18c55b0909'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:21.588768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6300acec-635d-4899-b6b5-b9f37abadf11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebea1b500dd44a9888d5cd18c55b0909'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:21.637220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1098/2186] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:21 2022] DELETE /identity/v3/users/ebea1b500dd44a9888d5cd18c55b0909 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:02:21.642903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277d314-5bf4-4dc3-993a-3d1cc833348a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:21.643221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277d314-5bf4-4dc3-993a-3d1cc833348a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:21.643506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b277d314-5bf4-4dc3-993a-3d1cc833348a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:21.682822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b277d314-5bf4-4dc3-993a-3d1cc833348a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:21.685629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b277d314-5bf4-4dc3-993a-3d1cc833348a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:21.713683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4875cac-936b-4871-9e05-70b15b278819 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:21.714438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4875cac-936b-4871-9e05-70b15b278819 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:21.714990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4875cac-936b-4871-9e05-70b15b278819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:21.738194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-a4875cac-936b-4871-9e05-70b15b278819 None None] MFA Rules not processed for user `3cc9d727a1e044a397b8b8b9883d5dcb`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:21.739987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a4875cac-936b-4871-9e05-70b15b278819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:21.783073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1089/2187] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:21.796282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277d314-5bf4-4dc3-993a-3d1cc833348a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:21.798465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b277d314-5bf4-4dc3-993a-3d1cc833348a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:21.825622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:21.860293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1099/2188] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:02:21.938189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b277d314-5bf4-4dc3-993a-3d1cc833348a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:21.947628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84868091-8dd3-4536-9c76-856ac4a3ae41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:21.949585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84868091-8dd3-4536-9c76-856ac4a3ae41 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:21.950026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84868091-8dd3-4536-9c76-856ac4a3ae41 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:21.950473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84868091-8dd3-4536-9c76-856ac4a3ae41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:21.993283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84868091-8dd3-4536-9c76-856ac4a3ae41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:21.995054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84868091-8dd3-4536-9c76-856ac4a3ae41 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:22.006561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1090/2189] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:02:22.112720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4875cac-936b-4871-9e05-70b15b278819 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:22.114278 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a4875cac-936b-4871-9e05-70b15b278819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:22.143065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4875cac-936b-4871-9e05-70b15b278819 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:22.153525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:22.162360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:22.162788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:22.163240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] PATH_INFO: `/v3/projects/80394175293a4901862b2cfacc09b18d` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:22.165990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=80394175293a4901862b2cfacc09b18d)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:22.168259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:22.333169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80394175293a4901862b2cfacc09b18d', 'request_id': 'req-d4e98765-5b66-4749-805b-09fd84cb5b1e'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.339612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80394175293a4901862b2cfacc09b18d', 'request_id': 'req-d4e98765-5b66-4749-805b-09fd84cb5b1e'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.346658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80394175293a4901862b2cfacc09b18d', 'request_id': 'req-d4e98765-5b66-4749-805b-09fd84cb5b1e'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.413369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] The token cache is being invalidate because project 80394175293a4901862b2cfacc09b18d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:22.413743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.414317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.416109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1100/2190] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:22 2022] DELETE /identity/v3/projects/80394175293a4901862b2cfacc09b18d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:22.420146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84868091-8dd3-4536-9c76-856ac4a3ae41 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:22.442257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84868091-8dd3-4536-9c76-856ac4a3ae41 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:22.463887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:22.467571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:22.549964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:22.551222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:22.552728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:22.553165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] PATH_INFO: `/v3/users/49961f8a893145fe872093de6d232450` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:22.557450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=49961f8a893145fe872093de6d232450)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:22.570140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:22.639277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e98765-5b66-4749-805b-09fd84cb5b1e admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:22.647730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:22.648407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:22.648700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:22.648960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] PATH_INFO: `/v3/projects/4f3f7cf061114254b3a440f64b16d803` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:22.650129 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3f7cf061114254b3a440f64b16d803)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:22.652549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:22.726134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:22.850705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49961f8a893145fe872093de6d232450'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.851736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49961f8a893145fe872093de6d232450'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.852552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49961f8a893145fe872093de6d232450'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.861708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3f7cf061114254b3a440f64b16d803', 'request_id': 'req-448312f2-35c5-40aa-9744-33d548e9f621'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.871252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3f7cf061114254b3a440f64b16d803', 'request_id': 'req-448312f2-35c5-40aa-9744-33d548e9f621'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.878397 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3f7cf061114254b3a440f64b16d803', 'request_id': 'req-448312f2-35c5-40aa-9744-33d548e9f621'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.965803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49961f8a893145fe872093de6d232450'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:22.984462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49961f8a893145fe872093de6d232450'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:23.017201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1091/2191] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:22 2022] DELETE /identity/v3/users/49961f8a893145fe872093de6d232450 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:23.069994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] The token cache is being invalidate because project 4f3f7cf061114254b3a440f64b16d803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:23.070307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:23.070988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-448312f2-35c5-40aa-9744-33d548e9f621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:23.074555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1101/2192] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:22 2022] DELETE /identity/v3/projects/4f3f7cf061114254b3a440f64b16d803 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:23.556762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:23.556762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:23.639328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16088a1c-5480-44db-bfaa-71205e6196ce admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:23.644075 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:23.644703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:23.644985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:23.645256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] PATH_INFO: `/v3/projects/229610c5441448e68ce3a3ed443727e2` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:23.646368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=229610c5441448e68ce3a3ed443727e2)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:23.648294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:23.776767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '229610c5441448e68ce3a3ed443727e2', 'request_id': 'req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:23.783848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '229610c5441448e68ce3a3ed443727e2', 'request_id': 'req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:23.857175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '229610c5441448e68ce3a3ed443727e2', 'request_id': 'req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:23.869908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] The token cache is being invalidate because project 229610c5441448e68ce3a3ed443727e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:23.870287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:23.871115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-821556c8-9f74-492b-aa0a-5d71b7a0f1ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:23.872743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1092/2193] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:23 2022] DELETE /identity/v3/projects/229610c5441448e68ce3a3ed443727e2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:23.877862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a453839-e0ce-491f-82c0-9c175fd583fb None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:23.878420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a453839-e0ce-491f-82c0-9c175fd583fb None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:23.878551 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a453839-e0ce-491f-82c0-9c175fd583fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:23.886303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:23.899386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-3a453839-e0ce-491f-82c0-9c175fd583fb None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:23.900601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3a453839-e0ce-491f-82c0-9c175fd583fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:23.973282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1102/2194] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:02:23.978716 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561389b-7e80-4160-8f8f-6e599f3f21d7 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:23.979177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561389b-7e80-4160-8f8f-6e599f3f21d7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:23.983136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561389b-7e80-4160-8f8f-6e599f3f21d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:24.003153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e561389b-7e80-4160-8f8f-6e599f3f21d7 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:24.004342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e561389b-7e80-4160-8f8f-6e599f3f21d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:24.012219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1093/2195] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:24.016486 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a453839-e0ce-491f-82c0-9c175fd583fb None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:24.026300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a453839-e0ce-491f-82c0-9c175fd583fb None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:24.030926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf12766-db3f-4965-8ef0-8afb545caa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:24.031549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf12766-db3f-4965-8ef0-8afb545caa00 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:24.031830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf12766-db3f-4965-8ef0-8afb545caa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:24.032125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf12766-db3f-4965-8ef0-8afb545caa00 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:24.032766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf12766-db3f-4965-8ef0-8afb545caa00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:02:24.033065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf12766-db3f-4965-8ef0-8afb545caa00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:24.034683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf12766-db3f-4965-8ef0-8afb545caa00 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:24.047168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1103/2196] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:02:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:02:24.053760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561389b-7e80-4160-8f8f-6e599f3f21d7 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:24.063786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561389b-7e80-4160-8f8f-6e599f3f21d7 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:24.068603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a80c8-ebae-48f1-9725-b191d00c17e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:24.069208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a80c8-ebae-48f1-9725-b191d00c17e3 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:24.069523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a80c8-ebae-48f1-9725-b191d00c17e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:24.069794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a80c8-ebae-48f1-9725-b191d00c17e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:24.070428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a80c8-ebae-48f1-9725-b191d00c17e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:24.072502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a80c8-ebae-48f1-9725-b191d00c17e3 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:24.144604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1094/2197] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:02:24 2022] POST /identity/v3/projects => generated 376 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:02:24.150052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf12766-db3f-4965-8ef0-8afb545caa00 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:24.160585 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf12766-db3f-4965-8ef0-8afb545caa00 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:24.220871 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1936fe2-89de-43b7-bb93-39ea22a70640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:24.221484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1936fe2-89de-43b7-bb93-39ea22a70640 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:24.221754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1936fe2-89de-43b7-bb93-39ea22a70640 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:24.222020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1936fe2-89de-43b7-bb93-39ea22a70640 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:24.222649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1936fe2-89de-43b7-bb93-39ea22a70640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:24.229121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1936fe2-89de-43b7-bb93-39ea22a70640 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:24.307716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1104/2198] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:02:24 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:02:24.312006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a80c8-ebae-48f1-9725-b191d00c17e3 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:24.318548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:24.322212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a80c8-ebae-48f1-9725-b191d00c17e3 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:24.326931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6abe6b-d55a-424f-85c6-64e791cb0cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:24.327562 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6abe6b-d55a-424f-85c6-64e791cb0cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:24.327837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6abe6b-d55a-424f-85c6-64e791cb0cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:24.328105 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6abe6b-d55a-424f-85c6-64e791cb0cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:24.328741 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6abe6b-d55a-424f-85c6-64e791cb0cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:24.330424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6abe6b-d55a-424f-85c6-64e791cb0cc6 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:24.340284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1095/2199] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:24 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:02:24.344854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1936fe2-89de-43b7-bb93-39ea22a70640 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:24.355078 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1936fe2-89de-43b7-bb93-39ea22a70640 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:24.360763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11dc07e-290e-4f70-9df8-bfe0372184dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:24.362765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dc07e-290e-4f70-9df8-bfe0372184dc admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:24.362765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dc07e-290e-4f70-9df8-bfe0372184dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:24.362765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11dc07e-290e-4f70-9df8-bfe0372184dc admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:24.362765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11dc07e-290e-4f70-9df8-bfe0372184dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:24.364147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11dc07e-290e-4f70-9df8-bfe0372184dc admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:24.375154 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1105/2200] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:24 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:02:24.379182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6abe6b-d55a-424f-85c6-64e791cb0cc6 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:24.389626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6abe6b-d55a-424f-85c6-64e791cb0cc6 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:24.394340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44effd4e-ca40-496e-bdf9-c7ed64199e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:24.397193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44effd4e-ca40-496e-bdf9-c7ed64199e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:24.397193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44effd4e-ca40-496e-bdf9-c7ed64199e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:24.397193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44effd4e-ca40-496e-bdf9-c7ed64199e1f admin admin] PATH_INFO: `/v3/projects/c3f5c4be639d4e1498ffdf20ba8e3e1b/users/c3160e3984794d13907e56cb3d88cea0/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:24.410419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44effd4e-ca40-496e-bdf9-c7ed64199e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f5c4be639d4e1498ffdf20ba8e3e1b, user_id=c3160e3984794d13907e56cb3d88cea0, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:24.413137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44effd4e-ca40-496e-bdf9-c7ed64199e1f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:24.502502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1096/2201] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:02:24 2022] PUT /identity/v3/projects/c3f5c4be639d4e1498ffdf20ba8e3e1b/users/c3160e3984794d13907e56cb3d88cea0/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:24.514151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071794d-73d9-4773-9e72-4984ea85f0ef None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:24.514534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071794d-73d9-4773-9e72-4984ea85f0ef None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:24.514809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071794d-73d9-4773-9e72-4984ea85f0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:24.542211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-3071794d-73d9-4773-9e72-4984ea85f0ef None None] MFA Rules not processed for user `c3160e3984794d13907e56cb3d88cea0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:24.543451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3071794d-73d9-4773-9e72-4984ea85f0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:24.603388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:24.608455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1106/2202] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:24.619856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44effd4e-ca40-496e-bdf9-c7ed64199e1f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:24.621582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-44effd4e-ca40-496e-bdf9-c7ed64199e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:24.679308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44effd4e-ca40-496e-bdf9-c7ed64199e1f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:24.682955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6579ae6-a3ed-43b3-b635-e4720db04d9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:24.683604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6579ae6-a3ed-43b3-b635-e4720db04d9c service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:24.683924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6579ae6-a3ed-43b3-b635-e4720db04d9c service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:24.684197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6579ae6-a3ed-43b3-b635-e4720db04d9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:24.692506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6579ae6-a3ed-43b3-b635-e4720db04d9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:24.693924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6579ae6-a3ed-43b3-b635-e4720db04d9c service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:24.703114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1097/2203] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:24.846790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfb4de-f0dc-44eb-a217-11bcf48e1e03 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:24.847118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfb4de-f0dc-44eb-a217-11bcf48e1e03 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:24.847401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfb4de-f0dc-44eb-a217-11bcf48e1e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:24.867550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2fbfb4de-f0dc-44eb-a217-11bcf48e1e03 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:24.868788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2fbfb4de-f0dc-44eb-a217-11bcf48e1e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:24.924741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1107/2204] 10.222.0.105 () {58 vars in 972 bytes} [Wed Aug 24 05:02:24 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:24.939114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6579ae6-a3ed-43b3-b635-e4720db04d9c service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:24.939114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f6579ae6-a3ed-43b3-b635-e4720db04d9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:25.009984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6579ae6-a3ed-43b3-b635-e4720db04d9c service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:25.015167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297aa31-60ed-4d27-9147-04267a39abdc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:25.015787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297aa31-60ed-4d27-9147-04267a39abdc service swift] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:25.016065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297aa31-60ed-4d27-9147-04267a39abdc service swift] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:25.016338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2297aa31-60ed-4d27-9147-04267a39abdc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:25.025152 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297aa31-60ed-4d27-9147-04267a39abdc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:25.026754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2297aa31-60ed-4d27-9147-04267a39abdc service swift] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:25.036524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1098/2205] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:02:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:02:28.797835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfb4de-f0dc-44eb-a217-11bcf48e1e03 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:28.800965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2fbfb4de-f0dc-44eb-a217-11bcf48e1e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:28.992823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfb4de-f0dc-44eb-a217-11bcf48e1e03 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:28.998338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:28.999558 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:28.999971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:29.000245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] PATH_INFO: `/v3/users/946a7533230b446f8ff34a0a51d4c2f2` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:29.001459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=946a7533230b446f8ff34a0a51d4c2f2)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:29.003421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:29.219272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '946a7533230b446f8ff34a0a51d4c2f2'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:29.220030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '946a7533230b446f8ff34a0a51d4c2f2'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:29.220651 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '946a7533230b446f8ff34a0a51d4c2f2'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:29.291561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '946a7533230b446f8ff34a0a51d4c2f2'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:29.322799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-2355382d-df9f-4336-b37a-ab67af971268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '946a7533230b446f8ff34a0a51d4c2f2'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:29.336104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1108/2206] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:28 2022] DELETE /identity/v3/users/946a7533230b446f8ff34a0a51d4c2f2 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:29.664552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297aa31-60ed-4d27-9147-04267a39abdc service swift] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:29.690626 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2297aa31-60ed-4d27-9147-04267a39abdc service swift] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:29.749818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:29.750451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:29.750734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:29.751008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] PATH_INFO: `/v3/projects/6458afad3a654baebc09ce00a9d695bb` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:29.752247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6458afad3a654baebc09ce00a9d695bb)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:29.754224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:29.756543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:29.856509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6458afad3a654baebc09ce00a9d695bb', 'request_id': 'req-daf52971-ae68-44df-b61b-c9fdbe72d421'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:29.863223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6458afad3a654baebc09ce00a9d695bb', 'request_id': 'req-daf52971-ae68-44df-b61b-c9fdbe72d421'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:29.936366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6458afad3a654baebc09ce00a9d695bb', 'request_id': 'req-daf52971-ae68-44df-b61b-c9fdbe72d421'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:29.947769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] The token cache is being invalidate because project 6458afad3a654baebc09ce00a9d695bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:29.947988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:29.948611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-daf52971-ae68-44df-b61b-c9fdbe72d421 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:29.950092 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1099/2207] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:29 2022] DELETE /identity/v3/projects/6458afad3a654baebc09ce00a9d695bb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:29.955032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7e27b-7290-4001-82aa-4bd893fefa4a None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:29.955713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7e27b-7290-4001-82aa-4bd893fefa4a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:29.955713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7e27b-7290-4001-82aa-4bd893fefa4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:29.976589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-aee7e27b-7290-4001-82aa-4bd893fefa4a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:29.981097 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-aee7e27b-7290-4001-82aa-4bd893fefa4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:29.995515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:30.040193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1109/2208] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:30.048473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752a396-2435-4f3e-955b-a675a942e099 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.048762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752a396-2435-4f3e-955b-a675a942e099 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.049038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752a396-2435-4f3e-955b-a675a942e099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.069900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9752a396-2435-4f3e-955b-a675a942e099 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:30.071167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9752a396-2435-4f3e-955b-a675a942e099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:30.093243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1100/2209] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:30.094802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7e27b-7290-4001-82aa-4bd893fefa4a None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:30.105124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7e27b-7290-4001-82aa-4bd893fefa4a None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:30.110711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e09e47-65d6-4e57-ab1c-4e65db18f8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:30.111360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e09e47-65d6-4e57-ab1c-4e65db18f8b7 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.111638 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e09e47-65d6-4e57-ab1c-4e65db18f8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.111911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e09e47-65d6-4e57-ab1c-4e65db18f8b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.115365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e09e47-65d6-4e57-ab1c-4e65db18f8b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:02:30.115698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e09e47-65d6-4e57-ab1c-4e65db18f8b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:30.117443 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e09e47-65d6-4e57-ab1c-4e65db18f8b7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:30.130766 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1110/2210] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:02:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:02:30.134887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752a396-2435-4f3e-955b-a675a942e099 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:30.144318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752a396-2435-4f3e-955b-a675a942e099 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:30.149724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181d220-226b-44db-8d76-130b31979239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:30.150336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181d220-226b-44db-8d76-130b31979239 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.150611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181d220-226b-44db-8d76-130b31979239 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.150919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4181d220-226b-44db-8d76-130b31979239 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.151581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181d220-226b-44db-8d76-130b31979239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:30.153487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4181d220-226b-44db-8d76-130b31979239 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:30.168774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e09e47-65d6-4e57-ab1c-4e65db18f8b7 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:30.170618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-17e09e47-65d6-4e57-ab1c-4e65db18f8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:30.184725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e09e47-65d6-4e57-ab1c-4e65db18f8b7 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:30.193903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:30.194553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.194833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.195599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] PATH_INFO: `/v3/users/c3160e3984794d13907e56cb3d88cea0` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.196456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3160e3984794d13907e56cb3d88cea0)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:30.204351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:30.261079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1101/2211] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:02:30 2022] POST /identity/v3/projects => generated 372 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:02:30.262158 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181d220-226b-44db-8d76-130b31979239 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:30.264161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4181d220-226b-44db-8d76-130b31979239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:30.337041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4181d220-226b-44db-8d76-130b31979239 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:30.342320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c66e22-3f5f-4a0d-8d3e-e1fdb878941b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:30.343091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c66e22-3f5f-4a0d-8d3e-e1fdb878941b service placement] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.343672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c66e22-3f5f-4a0d-8d3e-e1fdb878941b service placement] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.344194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c66e22-3f5f-4a0d-8d3e-e1fdb878941b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.345939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c7c66e22-3f5f-4a0d-8d3e-e1fdb878941b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:30.357343 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c66e22-3f5f-4a0d-8d3e-e1fdb878941b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:30.359323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c66e22-3f5f-4a0d-8d3e-e1fdb878941b service placement] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:30.369259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1102/2212] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:30 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:30.370023 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c66e22-3f5f-4a0d-8d3e-e1fdb878941b service placement] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:30.400480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3160e3984794d13907e56cb3d88cea0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:30.401359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3160e3984794d13907e56cb3d88cea0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:30.402656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3160e3984794d13907e56cb3d88cea0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:30.431127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:30.443144 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c66e22-3f5f-4a0d-8d3e-e1fdb878941b service placement] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:30.447743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d59da-0fc4-495f-b700-1f22b40d0751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:30.448365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d59da-0fc4-495f-b700-1f22b40d0751 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.448625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d59da-0fc4-495f-b700-1f22b40d0751 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.448899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090d59da-0fc4-495f-b700-1f22b40d0751 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.449528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d59da-0fc4-495f-b700-1f22b40d0751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:30.450892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3160e3984794d13907e56cb3d88cea0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:30.451473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090d59da-0fc4-495f-b700-1f22b40d0751 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:30.464879 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fd99ac75-f5c8-497d-bee0-db90a5eb4f5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3160e3984794d13907e56cb3d88cea0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:30.477129 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1111/2213] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:30 2022] DELETE /identity/v3/users/c3160e3984794d13907e56cb3d88cea0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:02:30.481409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882f46f-2471-4361-b4c5-13796e07dbc5 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.481692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882f46f-2471-4361-b4c5-13796e07dbc5 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.481957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d882f46f-2471-4361-b4c5-13796e07dbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.506917 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d882f46f-2471-4361-b4c5-13796e07dbc5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:30.508144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d882f46f-2471-4361-b4c5-13796e07dbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:30.529601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1103/2214] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:02:30 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:02:30.531357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d59da-0fc4-495f-b700-1f22b40d0751 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:30.533115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-090d59da-0fc4-495f-b700-1f22b40d0751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:30.544898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090d59da-0fc4-495f-b700-1f22b40d0751 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:30.570534 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1112/2215] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:02:30.580946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882f46f-2471-4361-b4c5-13796e07dbc5 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:30.582526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d882f46f-2471-4361-b4c5-13796e07dbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:30.602793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2bbc2-5ac4-4b78-83e1-f5364b4bf261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:30.603437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2bbc2-5ac4-4b78-83e1-f5364b4bf261 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.603719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2bbc2-5ac4-4b78-83e1-f5364b4bf261 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.604050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2bbc2-5ac4-4b78-83e1-f5364b4bf261 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.604751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2bbc2-5ac4-4b78-83e1-f5364b4bf261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:30.606396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2bbc2-5ac4-4b78-83e1-f5364b4bf261 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:30.610407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:30.617065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1104/2216] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:30 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:02:30.621170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2bbc2-5ac4-4b78-83e1-f5364b4bf261 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:30.631490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2bbc2-5ac4-4b78-83e1-f5364b4bf261 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:30.636226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f55dd-eca1-4a5c-b11f-e4e3908f6609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:30.636813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f55dd-eca1-4a5c-b11f-e4e3908f6609 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.637093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f55dd-eca1-4a5c-b11f-e4e3908f6609 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.637369 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f55dd-eca1-4a5c-b11f-e4e3908f6609 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.638019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f55dd-eca1-4a5c-b11f-e4e3908f6609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:30.639905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f55dd-eca1-4a5c-b11f-e4e3908f6609 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:30.643700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d882f46f-2471-4361-b4c5-13796e07dbc5 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:30.649795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8b560-11b1-4ad7-9e40-c6af7fb588fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:30.650402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8b560-11b1-4ad7-9e40-c6af7fb588fb service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.650681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8b560-11b1-4ad7-9e40-c6af7fb588fb service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.655603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8b560-11b1-4ad7-9e40-c6af7fb588fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.655752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1105/2217] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:30 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:02:30.655752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f55dd-eca1-4a5c-b11f-e4e3908f6609 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:30.661091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8b560-11b1-4ad7-9e40-c6af7fb588fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:30.662519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8b560-11b1-4ad7-9e40-c6af7fb588fb service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:30.665736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f55dd-eca1-4a5c-b11f-e4e3908f6609 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:30.670622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086b122-ff75-41e8-8b31-2b4ad3b04279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:30.672294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086b122-ff75-41e8-8b31-2b4ad3b04279 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.672472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086b122-ff75-41e8-8b31-2b4ad3b04279 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.672683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086b122-ff75-41e8-8b31-2b4ad3b04279 admin admin] PATH_INFO: `/v3/projects/8eb89bbdeb68492c84d3063743a90f12/users/09da472f4563467982f3468022b56f35/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.675436 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1113/2218] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:02:30.688954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086b122-ff75-41e8-8b31-2b4ad3b04279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb89bbdeb68492c84d3063743a90f12, user_id=09da472f4563467982f3468022b56f35, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:30.692050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086b122-ff75-41e8-8b31-2b4ad3b04279 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:30.788310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1106/2219] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:02:30 2022] PUT /identity/v3/projects/8eb89bbdeb68492c84d3063743a90f12/users/09da472f4563467982f3468022b56f35/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:30.799716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbd9e3-c4ca-4cb6-86d6-b7e7421bcea6 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.800000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbd9e3-c4ca-4cb6-86d6-b7e7421bcea6 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.800293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbd9e3-c4ca-4cb6-86d6-b7e7421bcea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.828639 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-addbd9e3-c4ca-4cb6-86d6-b7e7421bcea6 None None] MFA Rules not processed for user `09da472f4563467982f3468022b56f35`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:30.829837 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-addbd9e3-c4ca-4cb6-86d6-b7e7421bcea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:30.896597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1114/2220] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:30.908324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086b122-ff75-41e8-8b31-2b4ad3b04279 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:30.910089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c086b122-ff75-41e8-8b31-2b4ad3b04279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:30.980331 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086b122-ff75-41e8-8b31-2b4ad3b04279 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:30.987760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ef6a8-9d2f-4fa3-83dd-3b28f76a5460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:30.988375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ef6a8-9d2f-4fa3-83dd-3b28f76a5460 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:30.988652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ef6a8-9d2f-4fa3-83dd-3b28f76a5460 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:30.988936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ef6a8-9d2f-4fa3-83dd-3b28f76a5460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:30.997432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ef6a8-9d2f-4fa3-83dd-3b28f76a5460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:30.998849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ef6a8-9d2f-4fa3-83dd-3b28f76a5460 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:31.009908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1107/2221] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:31.033409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbd9e3-c4ca-4cb6-86d6-b7e7421bcea6 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:31.098091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbd9e3-c4ca-4cb6-86d6-b7e7421bcea6 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:31.100139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ef6a8-9d2f-4fa3-83dd-3b28f76a5460 service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:31.107780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-190ef6a8-9d2f-4fa3-83dd-3b28f76a5460 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:31.108032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:31.108032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.108032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.108032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] PATH_INFO: `/v3/users/09da472f4563467982f3468022b56f35` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.109127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] RBAC: Authorizing `identity:delete_user(user_id=09da472f4563467982f3468022b56f35)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:31.109127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:31.125984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ef6a8-9d2f-4fa3-83dd-3b28f76a5460 service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:31.133261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:31.133470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.133817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.134096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] PATH_INFO: `/v3/projects/c3f5c4be639d4e1498ffdf20ba8e3e1b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.136241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f5c4be639d4e1498ffdf20ba8e3e1b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:31.138266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:31.275121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:31.281782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09da472f4563467982f3468022b56f35'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:31.282723 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09da472f4563467982f3468022b56f35'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:31.283710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09da472f4563467982f3468022b56f35'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:31.291381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f5c4be639d4e1498ffdf20ba8e3e1b', 'request_id': 'req-cbd17705-3edd-4f2c-acfe-92348edecd93'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:31.299420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f5c4be639d4e1498ffdf20ba8e3e1b', 'request_id': 'req-cbd17705-3edd-4f2c-acfe-92348edecd93'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:31.360354 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09da472f4563467982f3468022b56f35'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:31.363320 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:31.373750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-3908f941-cd41-4a6b-9127-6b0f8a6b9dde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09da472f4563467982f3468022b56f35'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:31.394063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1115/2222] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:31 2022] DELETE /identity/v3/users/09da472f4563467982f3468022b56f35 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:31.398296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbf425-5d9e-43f9-b2ef-b080783deb7b None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.398578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbf425-5d9e-43f9-b2ef-b080783deb7b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.398847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acbf425-5d9e-43f9-b2ef-b080783deb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.418866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5acbf425-5d9e-43f9-b2ef-b080783deb7b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:31.419951 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5acbf425-5d9e-43f9-b2ef-b080783deb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:31.427808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f5c4be639d4e1498ffdf20ba8e3e1b', 'request_id': 'req-cbd17705-3edd-4f2c-acfe-92348edecd93'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:31.440335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] The token cache is being invalidate because project c3f5c4be639d4e1498ffdf20ba8e3e1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:31.440614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:31.441329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cbd17705-3edd-4f2c-acfe-92348edecd93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:31.443045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1108/2223] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:31 2022] DELETE /identity/v3/projects/c3f5c4be639d4e1498ffdf20ba8e3e1b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:31.447393 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315677c-2fe4-47bb-950c-1b793d16f12f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.447673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315677c-2fe4-47bb-950c-1b793d16f12f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.447945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315677c-2fe4-47bb-950c-1b793d16f12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.468937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c315677c-2fe4-47bb-950c-1b793d16f12f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:31.470142 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c315677c-2fe4-47bb-950c-1b793d16f12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:31.482713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1116/2224] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:02:31.492631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbf425-5d9e-43f9-b2ef-b080783deb7b None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:31.494020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5acbf425-5d9e-43f9-b2ef-b080783deb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:31.546677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1109/2225] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:02:31.546677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855d21-d5bf-4096-a87a-f348adbebb79 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.546677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855d21-d5bf-4096-a87a-f348adbebb79 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.546677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2855d21-d5bf-4096-a87a-f348adbebb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.554733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acbf425-5d9e-43f9-b2ef-b080783deb7b None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:31.558448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af32af-3688-4d39-b035-a4fbca1f4c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:31.559114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af32af-3688-4d39-b035-a4fbca1f4c17 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.559389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af32af-3688-4d39-b035-a4fbca1f4c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.559654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af32af-3688-4d39-b035-a4fbca1f4c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.564028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-c2855d21-d5bf-4096-a87a-f348adbebb79 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:31.565241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c2855d21-d5bf-4096-a87a-f348adbebb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:31.571181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af32af-3688-4d39-b035-a4fbca1f4c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:31.571181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1af32af-3688-4d39-b035-a4fbca1f4c17 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:31.573339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1110/2226] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:02:31.578065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2855d21-d5bf-4096-a87a-f348adbebb79 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:31.581188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1117/2227] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:02:31.586998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2855d21-d5bf-4096-a87a-f348adbebb79 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:31.591642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17138e31-da94-4b5b-8d8d-3fc28fb977f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:31.592249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17138e31-da94-4b5b-8d8d-3fc28fb977f1 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.592516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17138e31-da94-4b5b-8d8d-3fc28fb977f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.592807 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17138e31-da94-4b5b-8d8d-3fc28fb977f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.594005 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17138e31-da94-4b5b-8d8d-3fc28fb977f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:02:31.594318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17138e31-da94-4b5b-8d8d-3fc28fb977f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:31.595938 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17138e31-da94-4b5b-8d8d-3fc28fb977f1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:31.612678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1111/2228] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:02:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:02:31.613765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af32af-3688-4d39-b035-a4fbca1f4c17 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:31.623631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af32af-3688-4d39-b035-a4fbca1f4c17 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:31.629321 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312b3d3-c98e-4a18-bfac-f2175083bd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:31.629922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312b3d3-c98e-4a18-bfac-f2175083bd60 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.630203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312b3d3-c98e-4a18-bfac-f2175083bd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.630476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312b3d3-c98e-4a18-bfac-f2175083bd60 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.631142 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312b3d3-c98e-4a18-bfac-f2175083bd60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:31.633083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312b3d3-c98e-4a18-bfac-f2175083bd60 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:31.743938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1118/2229] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:02:31 2022] POST /identity/v3/projects => generated 366 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:02:31.747495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17138e31-da94-4b5b-8d8d-3fc28fb977f1 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:31.757800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17138e31-da94-4b5b-8d8d-3fc28fb977f1 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:31.815940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a1fdc-4eed-4e1d-9d13-c8c41963d807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:31.816557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a1fdc-4eed-4e1d-9d13-c8c41963d807 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.816833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a1fdc-4eed-4e1d-9d13-c8c41963d807 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.817103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a1fdc-4eed-4e1d-9d13-c8c41963d807 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.817727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a1fdc-4eed-4e1d-9d13-c8c41963d807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:31.819858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a1fdc-4eed-4e1d-9d13-c8c41963d807 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:31.919664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1112/2230] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:02:31 2022] POST /identity/v3/users => generated 284 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:02:31.924790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312b3d3-c98e-4a18-bfac-f2175083bd60 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:31.935453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312b3d3-c98e-4a18-bfac-f2175083bd60 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:31.940698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f5d22-453b-44df-aea5-952089d869d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:31.941303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f5d22-453b-44df-aea5-952089d869d7 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.941580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f5d22-453b-44df-aea5-952089d869d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.944392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f5d22-453b-44df-aea5-952089d869d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.944392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f5d22-453b-44df-aea5-952089d869d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:31.944392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f5d22-453b-44df-aea5-952089d869d7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:31.954749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1119/2231] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:31 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:02:31.958818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a1fdc-4eed-4e1d-9d13-c8c41963d807 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:31.969688 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a1fdc-4eed-4e1d-9d13-c8c41963d807 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:31.974463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce633a99-e21a-4c74-9650-3e9a62fe910f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:31.975145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce633a99-e21a-4c74-9650-3e9a62fe910f admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:31.975425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce633a99-e21a-4c74-9650-3e9a62fe910f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:31.975696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce633a99-e21a-4c74-9650-3e9a62fe910f admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:31.976329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce633a99-e21a-4c74-9650-3e9a62fe910f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:31.977910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce633a99-e21a-4c74-9650-3e9a62fe910f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:31.990148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1113/2232] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:31 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:02:31.994292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f5d22-453b-44df-aea5-952089d869d7 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:32.004720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce633a99-e21a-4c74-9650-3e9a62fe910f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:32.006435 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f5d22-453b-44df-aea5-952089d869d7 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:32.006757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ce633a99-e21a-4c74-9650-3e9a62fe910f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:32.011695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44edcb-ef08-47ca-b229-5194c3ae83c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:32.012113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44edcb-ef08-47ca-b229-5194c3ae83c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.012391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44edcb-ef08-47ca-b229-5194c3ae83c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.012720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44edcb-ef08-47ca-b229-5194c3ae83c1 admin admin] PATH_INFO: `/v3/projects/819c588595a9430d9e5776f1df315982/users/536f98b5f9b14fed91a46098ca0b09ef/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.020944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce633a99-e21a-4c74-9650-3e9a62fe910f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:32.027961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:32.028970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.029184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44edcb-ef08-47ca-b229-5194c3ae83c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=819c588595a9430d9e5776f1df315982, user_id=536f98b5f9b14fed91a46098ca0b09ef, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:32.029624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.030698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] PATH_INFO: `/v3/projects/8eb89bbdeb68492c84d3063743a90f12` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.032403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44edcb-ef08-47ca-b229-5194c3ae83c1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:32.032753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb89bbdeb68492c84d3063743a90f12)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:32.034734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:32.060551 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:32.097446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1120/2233] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:02:31 2022] PUT /identity/v3/projects/819c588595a9430d9e5776f1df315982/users/536f98b5f9b14fed91a46098ca0b09ef/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:32.107688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8709d0-837f-430b-a5af-576d2529a4f4 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.107973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8709d0-837f-430b-a5af-576d2529a4f4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.109073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8709d0-837f-430b-a5af-576d2529a4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.136521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-9d8709d0-837f-430b-a5af-576d2529a4f4 None None] MFA Rules not processed for user `536f98b5f9b14fed91a46098ca0b09ef`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:32.137765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9d8709d0-837f-430b-a5af-576d2529a4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:32.138976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb89bbdeb68492c84d3063743a90f12', 'request_id': 'req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:32.146126 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb89bbdeb68492c84d3063743a90f12', 'request_id': 'req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:32.146600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:32.216366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb89bbdeb68492c84d3063743a90f12', 'request_id': 'req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:32.217129 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1121/2234] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:02:32.230816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8709d0-837f-430b-a5af-576d2529a4f4 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:32.232208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9d8709d0-837f-430b-a5af-576d2529a4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:32.235783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] The token cache is being invalidate because project 8eb89bbdeb68492c84d3063743a90f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:32.236181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:32.236932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c8b9503b-ddf7-4b3f-9e4e-1162ea9cb59e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:32.238794 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1114/2235] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:32 2022] DELETE /identity/v3/projects/8eb89bbdeb68492c84d3063743a90f12 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:02:32.243569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9768cd-53ba-42e4-93cc-8ad17cb2158a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.243857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9768cd-53ba-42e4-93cc-8ad17cb2158a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.244132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9768cd-53ba-42e4-93cc-8ad17cb2158a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.272192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6c9768cd-53ba-42e4-93cc-8ad17cb2158a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:32.273603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6c9768cd-53ba-42e4-93cc-8ad17cb2158a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:32.299592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8709d0-837f-430b-a5af-576d2529a4f4 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:32.303377 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb87f63-20cf-4ec8-9f24-b66b7baeb807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:32.304096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb87f63-20cf-4ec8-9f24-b66b7baeb807 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.304379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb87f63-20cf-4ec8-9f24-b66b7baeb807 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.304647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb87f63-20cf-4ec8-9f24-b66b7baeb807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.306572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5bb87f63-20cf-4ec8-9f24-b66b7baeb807 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:32.324513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb87f63-20cf-4ec8-9f24-b66b7baeb807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:32.326516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb87f63-20cf-4ec8-9f24-b66b7baeb807 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:32.340360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1122/2236] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:32.354807 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1115/2237] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:32.363395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32736313-75ca-4a14-8ef7-2398c7986b69 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.363998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32736313-75ca-4a14-8ef7-2398c7986b69 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.364483 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32736313-75ca-4a14-8ef7-2398c7986b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.389790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-32736313-75ca-4a14-8ef7-2398c7986b69 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:32.391089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-32736313-75ca-4a14-8ef7-2398c7986b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:32.402327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1123/2238] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:32.403606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9768cd-53ba-42e4-93cc-8ad17cb2158a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:32.413574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9768cd-53ba-42e4-93cc-8ad17cb2158a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:32.418547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a2ed7-ba79-4b81-a529-31f552b0f530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:32.419201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2ed7-ba79-4b81-a529-31f552b0f530 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.419526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2ed7-ba79-4b81-a529-31f552b0f530 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.419803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2ed7-ba79-4b81-a529-31f552b0f530 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.420453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a2ed7-ba79-4b81-a529-31f552b0f530 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:02:32.421625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a2ed7-ba79-4b81-a529-31f552b0f530 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:32.424313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a2ed7-ba79-4b81-a529-31f552b0f530 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:32.441213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1116/2239] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:02:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:02:32.442309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32736313-75ca-4a14-8ef7-2398c7986b69 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:32.454479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32736313-75ca-4a14-8ef7-2398c7986b69 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:32.462116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6579df-27e3-4096-818f-7aed4f79c872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:32.462733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6579df-27e3-4096-818f-7aed4f79c872 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.463653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6579df-27e3-4096-818f-7aed4f79c872 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.463932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6579df-27e3-4096-818f-7aed4f79c872 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.464580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6579df-27e3-4096-818f-7aed4f79c872 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:32.467010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6579df-27e3-4096-818f-7aed4f79c872 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:32.602154 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1124/2240] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:02:32 2022] POST /identity/v3/projects => generated 398 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:02:32.606453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a2ed7-ba79-4b81-a529-31f552b0f530 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:32.617925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a2ed7-ba79-4b81-a529-31f552b0f530 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:32.688270 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c704c-c2a1-48a9-bc7a-1d05bfdff743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:32.688876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c704c-c2a1-48a9-bc7a-1d05bfdff743 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.689211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c704c-c2a1-48a9-bc7a-1d05bfdff743 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.689483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c704c-c2a1-48a9-bc7a-1d05bfdff743 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.690111 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c704c-c2a1-48a9-bc7a-1d05bfdff743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:32.692061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c704c-c2a1-48a9-bc7a-1d05bfdff743 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:32.828708 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1117/2241] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:02:32 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:02:32.832855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6579df-27e3-4096-818f-7aed4f79c872 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:32.843458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6579df-27e3-4096-818f-7aed4f79c872 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:32.848376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951f208-635a-4b42-bb94-3dae9c8b1584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:32.849012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951f208-635a-4b42-bb94-3dae9c8b1584 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.849310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951f208-635a-4b42-bb94-3dae9c8b1584 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.849635 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951f208-635a-4b42-bb94-3dae9c8b1584 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.850354 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951f208-635a-4b42-bb94-3dae9c8b1584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:32.851947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951f208-635a-4b42-bb94-3dae9c8b1584 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:32.864186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1125/2242] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:32 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:02:32.868135 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c704c-c2a1-48a9-bc7a-1d05bfdff743 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:32.882284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c704c-c2a1-48a9-bc7a-1d05bfdff743 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:32.887553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2fcb6f-79cc-4e1d-ad8c-d14e950f106a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:32.888287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fcb6f-79cc-4e1d-ad8c-d14e950f106a admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.888744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fcb6f-79cc-4e1d-ad8c-d14e950f106a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.889161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2fcb6f-79cc-4e1d-ad8c-d14e950f106a admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.890380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fcb6f-79cc-4e1d-ad8c-d14e950f106a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:32.892150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2fcb6f-79cc-4e1d-ad8c-d14e950f106a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:32.906793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1118/2243] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:02:32 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:02:32.910941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951f208-635a-4b42-bb94-3dae9c8b1584 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:32.922294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951f208-635a-4b42-bb94-3dae9c8b1584 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:32.927096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dea847-a6e5-40f1-99ff-b5f721c8b4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:32.927748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dea847-a6e5-40f1-99ff-b5f721c8b4f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:32.928030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dea847-a6e5-40f1-99ff-b5f721c8b4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:32.928300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dea847-a6e5-40f1-99ff-b5f721c8b4f7 admin admin] PATH_INFO: `/v3/projects/d975db42d3c648ff91559a85449b1456/users/e37aae99d3ad42b7b015ebe9bdb27101/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:32.943367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dea847-a6e5-40f1-99ff-b5f721c8b4f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d975db42d3c648ff91559a85449b1456, user_id=e37aae99d3ad42b7b015ebe9bdb27101, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:32.946121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dea847-a6e5-40f1-99ff-b5f721c8b4f7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:33.023218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1126/2244] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:02:32 2022] PUT /identity/v3/projects/d975db42d3c648ff91559a85449b1456/users/e37aae99d3ad42b7b015ebe9bdb27101/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:02:33.035285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a4e9a-10c5-415b-a637-36d4dd45ef86 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:33.035566 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a4e9a-10c5-415b-a637-36d4dd45ef86 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:33.035834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a4e9a-10c5-415b-a637-36d4dd45ef86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:33.063327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-221a4e9a-10c5-415b-a637-36d4dd45ef86 None None] MFA Rules not processed for user `e37aae99d3ad42b7b015ebe9bdb27101`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:33.064553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-221a4e9a-10c5-415b-a637-36d4dd45ef86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:33.069466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:33.129445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1119/2245] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:33 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:02:33.143414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dea847-a6e5-40f1-99ff-b5f721c8b4f7 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:33.145199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-24dea847-a6e5-40f1-99ff-b5f721c8b4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:33.219674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:33.223564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dea847-a6e5-40f1-99ff-b5f721c8b4f7 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:33.227281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16712b-fd68-4626-a05e-61fbd1a3df2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:33.227895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16712b-fd68-4626-a05e-61fbd1a3df2f service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:33.228179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16712b-fd68-4626-a05e-61fbd1a3df2f service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:33.228453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16712b-fd68-4626-a05e-61fbd1a3df2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:33.237241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16712b-fd68-4626-a05e-61fbd1a3df2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:33.238671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16712b-fd68-4626-a05e-61fbd1a3df2f service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:33.247512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1127/2246] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:33 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:02:43.531332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:43.599395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=24 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:45.338466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a4e9a-10c5-415b-a637-36d4dd45ef86 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:45.420303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a4e9a-10c5-415b-a637-36d4dd45ef86 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:45.425825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:45.426521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:45.426802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:45.427108 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] PATH_INFO: `/v3/users/e37aae99d3ad42b7b015ebe9bdb27101` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:45.428295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e37aae99d3ad42b7b015ebe9bdb27101)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:45.430405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:45.648003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e37aae99d3ad42b7b015ebe9bdb27101'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:45.648770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e37aae99d3ad42b7b015ebe9bdb27101'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:45.649388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e37aae99d3ad42b7b015ebe9bdb27101'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:45.741405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e37aae99d3ad42b7b015ebe9bdb27101'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:45.755416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e37aae99d3ad42b7b015ebe9bdb27101'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:45.768908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1120/2247] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:02:45 2022] DELETE /identity/v3/users/e37aae99d3ad42b7b015ebe9bdb27101 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:45.774063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab86d5-c733-47bc-80ba-f45eda46e339 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:45.774461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab86d5-c733-47bc-80ba-f45eda46e339 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:45.774669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab86d5-c733-47bc-80ba-f45eda46e339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:45.793891 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-70ab86d5-c733-47bc-80ba-f45eda46e339 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:02:45.795126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-70ab86d5-c733-47bc-80ba-f45eda46e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:45.857888 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1128/2248] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:02:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:02:45.868150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:45.869728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:45.932250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02464e7-7baf-4f70-83ea-e111e48d9c40 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:45.936207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2452d-a841-404e-881c-9d104ed58143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:45.937518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2452d-a841-404e-881c-9d104ed58143 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:45.937795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2452d-a841-404e-881c-9d104ed58143 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:45.938113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2452d-a841-404e-881c-9d104ed58143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:45.949010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2452d-a841-404e-881c-9d104ed58143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:45.950441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2452d-a841-404e-881c-9d104ed58143 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:45.960553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1121/2249] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:02:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:02:46.344543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab86d5-c733-47bc-80ba-f45eda46e339 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:02:46.345979 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-70ab86d5-c733-47bc-80ba-f45eda46e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:02:46.358691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab86d5-c733-47bc-80ba-f45eda46e339 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:02:46.363875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:02:46.364513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:02:46.364786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:02:46.365052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] PATH_INFO: `/v3/projects/d975db42d3c648ff91559a85449b1456` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:02:46.366200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d975db42d3c648ff91559a85449b1456)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:02:46.368128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:02:46.470811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd975db42d3c648ff91559a85449b1456', 'request_id': 'req-cde065b4-04c6-41dd-8205-e297b5462b78'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:46.489775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd975db42d3c648ff91559a85449b1456', 'request_id': 'req-cde065b4-04c6-41dd-8205-e297b5462b78'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:46.495782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd975db42d3c648ff91559a85449b1456', 'request_id': 'req-cde065b4-04c6-41dd-8205-e297b5462b78'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:46.565274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] The token cache is being invalidate because project d975db42d3c648ff91559a85449b1456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:02:46.565563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:46.566189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:02:46.567740 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1129/2250] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:02:46 2022] DELETE /identity/v3/projects/d975db42d3c648ff91559a85449b1456 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:02:55.955818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:02:56.561048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:03:06.686499 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e5f27-f602-4605-9684-11c1de2a9235 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:06.686994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e5f27-f602-4605-9684-11c1de2a9235 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:06.690606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e5f27-f602-4605-9684-11c1de2a9235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:06.707524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-555e5f27-f602-4605-9684-11c1de2a9235 None None] MFA Rules not processed for user `e45111c49aa74f76874d3a0c747034ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:03:06.710967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-555e5f27-f602-4605-9684-11c1de2a9235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:06.765228 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1122/2251] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:03:06 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:03:16.758424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=25 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:03:54.869980 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:54.871576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:54.939188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde065b4-04c6-41dd-8205-e297b5462b78 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:54.943805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:54.944439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:54.944724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:54.945012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] PATH_INFO: `/v3/users/536f98b5f9b14fed91a46098ca0b09ef` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:54.946191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=536f98b5f9b14fed91a46098ca0b09ef)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:54.948151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:55.130541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '536f98b5f9b14fed91a46098ca0b09ef'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:03:55.131231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '536f98b5f9b14fed91a46098ca0b09ef'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:03:55.131834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '536f98b5f9b14fed91a46098ca0b09ef'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:03:55.220483 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '536f98b5f9b14fed91a46098ca0b09ef'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:03:55.236467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '536f98b5f9b14fed91a46098ca0b09ef'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:03:55.249601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1130/2252] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:03:54 2022] DELETE /identity/v3/users/536f98b5f9b14fed91a46098ca0b09ef => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:03:55.254180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05029d-bdc7-47e3-a301-da2ee1ff134f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:55.254674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05029d-bdc7-47e3-a301-da2ee1ff134f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:55.254771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a05029d-bdc7-47e3-a301-da2ee1ff134f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:55.276111 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6a05029d-bdc7-47e3-a301-da2ee1ff134f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:03:55.277326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6a05029d-bdc7-47e3-a301-da2ee1ff134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:55.336882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1123/2253] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:03:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:03:55.347343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:55.349733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:55.412090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a44bdf-f34a-4b47-9467-816ebf64b1d4 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:55.415727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726d7cc-6757-4568-8a1d-14bdaeee3391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:55.416318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726d7cc-6757-4568-8a1d-14bdaeee3391 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:55.416590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726d7cc-6757-4568-8a1d-14bdaeee3391 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:55.416857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726d7cc-6757-4568-8a1d-14bdaeee3391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:55.426276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726d7cc-6757-4568-8a1d-14bdaeee3391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:55.427703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b726d7cc-6757-4568-8a1d-14bdaeee3391 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:55.437304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1131/2254] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:03:55 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:03:55.844552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05029d-bdc7-47e3-a301-da2ee1ff134f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:55.845979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6a05029d-bdc7-47e3-a301-da2ee1ff134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:55.859156 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a05029d-bdc7-47e3-a301-da2ee1ff134f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:55.864904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:55.865594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:55.865869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:55.866139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] PATH_INFO: `/v3/projects/819c588595a9430d9e5776f1df315982` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:55.867424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=819c588595a9430d9e5776f1df315982)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:55.870322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:56.009749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819c588595a9430d9e5776f1df315982', 'request_id': 'req-7e7ca972-3885-42e3-b9dc-152528356d34'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:03:56.016667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819c588595a9430d9e5776f1df315982', 'request_id': 'req-7e7ca972-3885-42e3-b9dc-152528356d34'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:03:56.128909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819c588595a9430d9e5776f1df315982', 'request_id': 'req-7e7ca972-3885-42e3-b9dc-152528356d34'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:03:56.140408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] The token cache is being invalidate because project 819c588595a9430d9e5776f1df315982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:03:56.140868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:03:56.141535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e7ca972-3885-42e3-b9dc-152528356d34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:03:56.143083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1124/2255] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:03:55 2022] DELETE /identity/v3/projects/819c588595a9430d9e5776f1df315982 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:03:56.148075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af20928-93a1-4b5c-9808-632c5ae0e505 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:56.148587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af20928-93a1-4b5c-9808-632c5ae0e505 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:56.149795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af20928-93a1-4b5c-9808-632c5ae0e505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:56.168838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-7af20928-93a1-4b5c-9808-632c5ae0e505 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:03:56.170971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-7af20928-93a1-4b5c-9808-632c5ae0e505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:56.190016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:03:56.237053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1132/2256] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:03:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:03:56.245298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68b2be-aa1e-46ff-8682-d6faed673233 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:56.247331 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68b2be-aa1e-46ff-8682-d6faed673233 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:56.247331 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68b2be-aa1e-46ff-8682-d6faed673233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:56.267329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ee68b2be-aa1e-46ff-8682-d6faed673233 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:03:56.268549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ee68b2be-aa1e-46ff-8682-d6faed673233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:56.276610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1125/2257] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:03:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:03:56.281485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af20928-93a1-4b5c-9808-632c5ae0e505 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:56.291777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af20928-93a1-4b5c-9808-632c5ae0e505 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:56.298410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e4c7c-41ed-4757-83e5-f5faa009de26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:56.299149 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e4c7c-41ed-4757-83e5-f5faa009de26 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:56.299495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e4c7c-41ed-4757-83e5-f5faa009de26 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:56.299834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43e4c7c-41ed-4757-83e5-f5faa009de26 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:56.300560 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e4c7c-41ed-4757-83e5-f5faa009de26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:03:56.300950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e4c7c-41ed-4757-83e5-f5faa009de26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:56.302620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43e4c7c-41ed-4757-83e5-f5faa009de26 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:56.315731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1133/2258] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:03:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:03:56.320411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68b2be-aa1e-46ff-8682-d6faed673233 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:56.329976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68b2be-aa1e-46ff-8682-d6faed673233 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:56.334793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a26e96-22e6-493e-bc9f-9242da8b1acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:56.335517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26e96-22e6-493e-bc9f-9242da8b1acf admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:56.335867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26e96-22e6-493e-bc9f-9242da8b1acf admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:56.336240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a26e96-22e6-493e-bc9f-9242da8b1acf admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:56.336965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a26e96-22e6-493e-bc9f-9242da8b1acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:56.338955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a26e96-22e6-493e-bc9f-9242da8b1acf admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:56.433770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1126/2259] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:03:56 2022] POST /identity/v3/projects => generated 374 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:03:56.438472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e4c7c-41ed-4757-83e5-f5faa009de26 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:56.448646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43e4c7c-41ed-4757-83e5-f5faa009de26 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:56.510613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e2467-e498-4537-9f2a-74fa81b16551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:56.511256 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e2467-e498-4537-9f2a-74fa81b16551 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:56.511542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e2467-e498-4537-9f2a-74fa81b16551 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:56.511818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e2467-e498-4537-9f2a-74fa81b16551 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:56.512452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e2467-e498-4537-9f2a-74fa81b16551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:56.514446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e2467-e498-4537-9f2a-74fa81b16551 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:56.637501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1134/2260] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:03:56 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:03:56.641480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a26e96-22e6-493e-bc9f-9242da8b1acf admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:56.651546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a26e96-22e6-493e-bc9f-9242da8b1acf admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:56.656874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43b60f-4de0-4f21-a625-6da236561be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:56.657470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43b60f-4de0-4f21-a625-6da236561be6 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:56.657750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43b60f-4de0-4f21-a625-6da236561be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:56.658024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43b60f-4de0-4f21-a625-6da236561be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:56.658658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43b60f-4de0-4f21-a625-6da236561be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:56.660282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43b60f-4de0-4f21-a625-6da236561be6 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:56.671159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1127/2261] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:03:56 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:03:56.675121 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e2467-e498-4537-9f2a-74fa81b16551 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:56.685087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e2467-e498-4537-9f2a-74fa81b16551 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:56.689941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02c004-c2a1-4f5f-9ce4-5c9ca4042483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:56.690533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02c004-c2a1-4f5f-9ce4-5c9ca4042483 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:56.690812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02c004-c2a1-4f5f-9ce4-5c9ca4042483 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:56.691122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02c004-c2a1-4f5f-9ce4-5c9ca4042483 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:56.691758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02c004-c2a1-4f5f-9ce4-5c9ca4042483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:56.693356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02c004-c2a1-4f5f-9ce4-5c9ca4042483 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:56.704075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1135/2262] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:03:56 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:03:56.708232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43b60f-4de0-4f21-a625-6da236561be6 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:56.718813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43b60f-4de0-4f21-a625-6da236561be6 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:56.726910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2689e-8562-4a37-892b-609fe9166207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:56.727595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2689e-8562-4a37-892b-609fe9166207 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:56.727871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2689e-8562-4a37-892b-609fe9166207 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:56.728137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2689e-8562-4a37-892b-609fe9166207 admin admin] PATH_INFO: `/v3/projects/f6a6379a0607459187075dfcd968d706/users/bfb6ab7a2f344a069010a3385d2fcf59/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:56.743809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2689e-8562-4a37-892b-609fe9166207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6a6379a0607459187075dfcd968d706, user_id=bfb6ab7a2f344a069010a3385d2fcf59, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:56.746419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2689e-8562-4a37-892b-609fe9166207 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:56.809478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1128/2263] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:03:56 2022] PUT /identity/v3/projects/f6a6379a0607459187075dfcd968d706/users/bfb6ab7a2f344a069010a3385d2fcf59/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:03:56.815379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee82ea-b019-4ff2-a5eb-949c4ed0fe8e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:56.815667 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee82ea-b019-4ff2-a5eb-949c4ed0fe8e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:56.815939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee82ea-b019-4ff2-a5eb-949c4ed0fe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:56.836400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-50ee82ea-b019-4ff2-a5eb-949c4ed0fe8e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:03:56.837622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-50ee82ea-b019-4ff2-a5eb-949c4ed0fe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:56.904393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1136/2264] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:03:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:03:56.912045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2689e-8562-4a37-892b-609fe9166207 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:56.913745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-47d2689e-8562-4a37-892b-609fe9166207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:56.936632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:03:56.972992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2689e-8562-4a37-892b-609fe9166207 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:56.976801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f45b7-40c0-4f8d-9f1c-e9c8f781b3b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:56.977401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f45b7-40c0-4f8d-9f1c-e9c8f781b3b8 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:56.977673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f45b7-40c0-4f8d-9f1c-e9c8f781b3b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:56.977940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f45b7-40c0-4f8d-9f1c-e9c8f781b3b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:56.987651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f45b7-40c0-4f8d-9f1c-e9c8f781b3b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:56.989044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f45b7-40c0-4f8d-9f1c-e9c8f781b3b8 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:56.998296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1129/2265] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:03:56 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:03:58.374652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472f838-f1f7-4801-8d10-32d4ba16ab87 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:58.375489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472f838-f1f7-4801-8d10-32d4ba16ab87 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:58.375763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472f838-f1f7-4801-8d10-32d4ba16ab87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:58.395345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c472f838-f1f7-4801-8d10-32d4ba16ab87 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:03:58.397658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c472f838-f1f7-4801-8d10-32d4ba16ab87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:58.422922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:03:58.463755 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1137/2266] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 05:03:58 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:03:58.591373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f45b7-40c0-4f8d-9f1c-e9c8f781b3b8 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:03:58.593801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-289f45b7-40c0-4f8d-9f1c-e9c8f781b3b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:03:58.673330 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f45b7-40c0-4f8d-9f1c-e9c8f781b3b8 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:03:58.678336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14c0ee-3f46-4e1d-a8db-8f4bce52527b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:03:58.678943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14c0ee-3f46-4e1d-a8db-8f4bce52527b service swift] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:03:58.679278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14c0ee-3f46-4e1d-a8db-8f4bce52527b service swift] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:03:58.679550 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14c0ee-3f46-4e1d-a8db-8f4bce52527b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:03:58.688447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14c0ee-3f46-4e1d-a8db-8f4bce52527b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:03:58.690031 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d14c0ee-3f46-4e1d-a8db-8f4bce52527b service swift] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:03:58.698516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1130/2267] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:03:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:04:04.152112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4d838-e2d5-4b82-b9e0-95f0e2cdced7 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:04.153898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4d838-e2d5-4b82-b9e0-95f0e2cdced7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:04.154457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4d838-e2d5-4b82-b9e0-95f0e2cdced7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:04.200079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b4d4d838-e2d5-4b82-b9e0-95f0e2cdced7 None None] MFA Rules not processed for user `bfb6ab7a2f344a069010a3385d2fcf59`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:04:04.203614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b4d4d838-e2d5-4b82-b9e0-95f0e2cdced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:04.360120 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14c0ee-3f46-4e1d-a8db-8f4bce52527b service swift] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:04.362083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1138/2268] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:04:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:04:04.375419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14c0ee-3f46-4e1d-a8db-8f4bce52527b service swift] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:04.390050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657d5aa-aa28-49e9-8583-7ce56445424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:04.390889 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657d5aa-aa28-49e9-8583-7ce56445424a admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:04.391276 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657d5aa-aa28-49e9-8583-7ce56445424a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:04.391642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657d5aa-aa28-49e9-8583-7ce56445424a admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:04.392368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657d5aa-aa28-49e9-8583-7ce56445424a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:04.394483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657d5aa-aa28-49e9-8583-7ce56445424a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:04.538894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1131/2269] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:04:04 2022] POST /identity/v3/projects => generated 374 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:04:04.548626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4d838-e2d5-4b82-b9e0-95f0e2cdced7 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:04.564112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4d838-e2d5-4b82-b9e0-95f0e2cdced7 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:04.656252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efa086-18cf-4ee7-9587-c4806e4f356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:04.657031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efa086-18cf-4ee7-9587-c4806e4f356a admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:04.657403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efa086-18cf-4ee7-9587-c4806e4f356a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:04.657938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efa086-18cf-4ee7-9587-c4806e4f356a admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:04.658532 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74efa086-18cf-4ee7-9587-c4806e4f356a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:04.660600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74efa086-18cf-4ee7-9587-c4806e4f356a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:04.751976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657d5aa-aa28-49e9-8583-7ce56445424a admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:04.752898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1139/2270] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:04:04 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:04:04.763752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657d5aa-aa28-49e9-8583-7ce56445424a admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:04.770929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ecce4-ecac-4350-b223-cd304e4c74d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:04.770929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ecce4-ecac-4350-b223-cd304e4c74d0 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:04.770929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ecce4-ecac-4350-b223-cd304e4c74d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:04.770929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ecce4-ecac-4350-b223-cd304e4c74d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:04.771584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ecce4-ecac-4350-b223-cd304e4c74d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:04.774009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ecce4-ecac-4350-b223-cd304e4c74d0 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:04.789881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1132/2271] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:04:04 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:04:04.791134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efa086-18cf-4ee7-9587-c4806e4f356a admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:04.813799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74efa086-18cf-4ee7-9587-c4806e4f356a admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:04.820763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf9493-c453-4453-a9f2-dd10b9d19fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:04.821463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf9493-c453-4453-a9f2-dd10b9d19fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:04.821744 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf9493-c453-4453-a9f2-dd10b9d19fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:04.822022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf9493-c453-4453-a9f2-dd10b9d19fb0 admin admin] PATH_INFO: `/v3/projects/cae251b8db4b41ed8a7c98aa77a40a49/users/7ca2dda5987649ae91db22bf20852355/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:04.839136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf9493-c453-4453-a9f2-dd10b9d19fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae251b8db4b41ed8a7c98aa77a40a49, user_id=7ca2dda5987649ae91db22bf20852355, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:04.842362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf9493-c453-4453-a9f2-dd10b9d19fb0 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:04.951885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1140/2272] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:04:04 2022] PUT /identity/v3/projects/cae251b8db4b41ed8a7c98aa77a40a49/users/7ca2dda5987649ae91db22bf20852355/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:04:04.952870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ecce4-ecac-4350-b223-cd304e4c74d0 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:04.967319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ecce4-ecac-4350-b223-cd304e4c74d0 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:05.055798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6675556-bd7f-4c54-865b-0140eeedad46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:05.056820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6675556-bd7f-4c54-865b-0140eeedad46 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:05.057322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6675556-bd7f-4c54-865b-0140eeedad46 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:05.057966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6675556-bd7f-4c54-865b-0140eeedad46 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:05.059426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6675556-bd7f-4c54-865b-0140eeedad46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:05.062508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6675556-bd7f-4c54-865b-0140eeedad46 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:05.075854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1133/2273] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:04:04 2022] GET /identity/v3/roles => generated 1427 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:04:05.079929 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf9493-c453-4453-a9f2-dd10b9d19fb0 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:05.093854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf9493-c453-4453-a9f2-dd10b9d19fb0 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:05.099872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e7f79-ad47-45a3-918e-228ba7a613b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:05.101043 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7f79-ad47-45a3-918e-228ba7a613b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:05.101646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7f79-ad47-45a3-918e-228ba7a613b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:05.101927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7f79-ad47-45a3-918e-228ba7a613b6 admin admin] PATH_INFO: `/v3/projects/cae251b8db4b41ed8a7c98aa77a40a49/users/7ca2dda5987649ae91db22bf20852355/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:05.112025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e7f79-ad47-45a3-918e-228ba7a613b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae251b8db4b41ed8a7c98aa77a40a49, user_id=7ca2dda5987649ae91db22bf20852355, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:05.116134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e7f79-ad47-45a3-918e-228ba7a613b6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:05.139799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1141/2274] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:04:05 2022] PUT /identity/v3/projects/cae251b8db4b41ed8a7c98aa77a40a49/users/7ca2dda5987649ae91db22bf20852355/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:04:13.064948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ff55a-591b-409b-bf4b-8657433700e1 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:13.065456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ff55a-591b-409b-bf4b-8657433700e1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:13.065567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ff55a-591b-409b-bf4b-8657433700e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:13.107492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-d47ff55a-591b-409b-bf4b-8657433700e1 None None] MFA Rules not processed for user `7ca2dda5987649ae91db22bf20852355`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:04:13.108658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d47ff55a-591b-409b-bf4b-8657433700e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:13.113432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:04:13.214561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1134/2275] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:04:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:04:13.229102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e7f79-ad47-45a3-918e-228ba7a613b6 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:13.230794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-695e7f79-ad47-45a3-918e-228ba7a613b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:13.252304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:04:13.315659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e7f79-ad47-45a3-918e-228ba7a613b6 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:13.321292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01fad41-3fad-4faa-85e0-df558344be38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:13.325359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fad41-3fad-4faa-85e0-df558344be38 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:13.325359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fad41-3fad-4faa-85e0-df558344be38 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:13.325359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01fad41-3fad-4faa-85e0-df558344be38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:13.382169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fad41-3fad-4faa-85e0-df558344be38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:13.383810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01fad41-3fad-4faa-85e0-df558344be38 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:13.393001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1142/2276] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:04:16.091628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917edf2-e548-410b-866b-b3c194c65622 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:16.092153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917edf2-e548-410b-866b-b3c194c65622 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:16.092153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917edf2-e548-410b-866b-b3c194c65622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:16.111381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8917edf2-e548-410b-866b-b3c194c65622 None None] MFA Rules not processed for user `bfb6ab7a2f344a069010a3385d2fcf59`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:04:16.112663 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8917edf2-e548-410b-866b-b3c194c65622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:16.123043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1135/2277] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:04:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:04:16.716027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a672f2-9dc7-400b-927a-b4e43c29c0df None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:16.716587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a672f2-9dc7-400b-927a-b4e43c29c0df None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:16.716587 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a672f2-9dc7-400b-927a-b4e43c29c0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:16.737696 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-64a672f2-9dc7-400b-927a-b4e43c29c0df None None] MFA Rules not processed for user `3cc9d727a1e044a397b8b8b9883d5dcb`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:04:16.738939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-64a672f2-9dc7-400b-927a-b4e43c29c0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:16.800698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1143/2278] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:04:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:04:19.406103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917edf2-e548-410b-866b-b3c194c65622 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:19.407617 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8917edf2-e548-410b-866b-b3c194c65622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:19.463593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917edf2-e548-410b-866b-b3c194c65622 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:19.467187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1e2c5-daec-489c-a5d8-d0eb59acf362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:19.467784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1e2c5-daec-489c-a5d8-d0eb59acf362 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:19.468063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1e2c5-daec-489c-a5d8-d0eb59acf362 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:19.468335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1e2c5-daec-489c-a5d8-d0eb59acf362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:19.470036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f3f1e2c5-daec-489c-a5d8-d0eb59acf362 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:19.483603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1e2c5-daec-489c-a5d8-d0eb59acf362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:19.485099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1e2c5-daec-489c-a5d8-d0eb59acf362 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:19.496531 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1136/2279] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:19 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:04:21.876228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a672f2-9dc7-400b-927a-b4e43c29c0df None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:21.886678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a672f2-9dc7-400b-927a-b4e43c29c0df None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:21.890258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1662201-5263-4f95-a10b-b1fa0c08b8ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:21.890857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1662201-5263-4f95-a10b-b1fa0c08b8ef service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:21.891165 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1662201-5263-4f95-a10b-b1fa0c08b8ef service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:21.891433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1662201-5263-4f95-a10b-b1fa0c08b8ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:21.893181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e1662201-5263-4f95-a10b-b1fa0c08b8ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:21.907992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1662201-5263-4f95-a10b-b1fa0c08b8ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:21.909402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1662201-5263-4f95-a10b-b1fa0c08b8ef service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:21.921309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1144/2280] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:21 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:04:23.823446 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4100c-fdc9-4959-bd96-398cc4f65d86 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:23.824130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4100c-fdc9-4959-bd96-398cc4f65d86 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:23.824130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4100c-fdc9-4959-bd96-398cc4f65d86 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:23.825524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1137/2281] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 05:04:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 05:04:23.831805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d53758-bd08-4e25-a297-209d7f45d47e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:23.832329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d53758-bd08-4e25-a297-209d7f45d47e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:23.832329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d53758-bd08-4e25-a297-209d7f45d47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:23.852287 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-e2d53758-bd08-4e25-a297-209d7f45d47e None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:04:23.853671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e2d53758-bd08-4e25-a297-209d7f45d47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:23.864985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1145/2282] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 05:04:23 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:04:23.875957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4100c-fdc9-4959-bd96-398cc4f65d86 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:23.878427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a5a4100c-fdc9-4959-bd96-398cc4f65d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:23.957935 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4100c-fdc9-4959-bd96-398cc4f65d86 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:23.961557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5cd7e-32a4-435d-8ff5-4b70784b6d4a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d62a4c07c4402ea5803f6e68fb1d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:23.962159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5cd7e-32a4-435d-8ff5-4b70784b6d4a service glance] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:23.962436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5cd7e-32a4-435d-8ff5-4b70784b6d4a service glance] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:23.962705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5cd7e-32a4-435d-8ff5-4b70784b6d4a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:23.972558 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5cd7e-32a4-435d-8ff5-4b70784b6d4a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:23.973963 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5cd7e-32a4-435d-8ff5-4b70784b6d4a service glance] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:23.984999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1138/2283] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:23 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:04:24.177220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2a04a-7a5c-4ab6-82a3-ea72f4a0e570 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:24.177501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2a04a-7a5c-4ab6-82a3-ea72f4a0e570 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:24.177772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2a04a-7a5c-4ab6-82a3-ea72f4a0e570 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:24.179166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1146/2284] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 05:04:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 05:04:24.184596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0f436-4542-4397-89fe-a357af5401bb None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:24.184880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0f436-4542-4397-89fe-a357af5401bb None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:24.185176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc0f436-4542-4397-89fe-a357af5401bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:24.204911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-dfc0f436-4542-4397-89fe-a357af5401bb None None] MFA Rules not processed for user `44d2b4015ac74b8b8d0a674419f34cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:04:24.206101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-dfc0f436-4542-4397-89fe-a357af5401bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:24.212717 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1139/2285] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 05:04:24 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:04:24.224540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2a04a-7a5c-4ab6-82a3-ea72f4a0e570 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:24.236259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2a04a-7a5c-4ab6-82a3-ea72f4a0e570 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:24.239844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe7f7f-d7df-4661-b6ce-5bd5122b5e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:24.240435 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe7f7f-d7df-4661-b6ce-5bd5122b5e13 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:24.240743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe7f7f-d7df-4661-b6ce-5bd5122b5e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:24.241018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe7f7f-d7df-4661-b6ce-5bd5122b5e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:24.248861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe7f7f-d7df-4661-b6ce-5bd5122b5e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:24.250273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe7f7f-d7df-4661-b6ce-5bd5122b5e13 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:24.257942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1147/2286] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:24 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:04:24.343585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db808d12-62e5-48c8-bf90-3edbf787c789 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:24.343868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db808d12-62e5-48c8-bf90-3edbf787c789 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:24.344140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db808d12-62e5-48c8-bf90-3edbf787c789 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:24.345488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1140/2287] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 05:04:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 05:04:24.350551 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075f0c-4dba-41cc-a778-ff15420f62cd None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:24.351384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075f0c-4dba-41cc-a778-ff15420f62cd None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:24.351658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b075f0c-4dba-41cc-a778-ff15420f62cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:24.371579 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-9b075f0c-4dba-41cc-a778-ff15420f62cd None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:04:24.372663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9b075f0c-4dba-41cc-a778-ff15420f62cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:24.383454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1148/2288] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 05:04:24 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:04:24.393970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db808d12-62e5-48c8-bf90-3edbf787c789 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:24.406812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db808d12-62e5-48c8-bf90-3edbf787c789 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:24.411472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af49bc-a63c-434a-8d5b-cfa66b8b4330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:24.412176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af49bc-a63c-434a-8d5b-cfa66b8b4330 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:24.412556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af49bc-a63c-434a-8d5b-cfa66b8b4330 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:24.412931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af49bc-a63c-434a-8d5b-cfa66b8b4330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:24.426706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af49bc-a63c-434a-8d5b-cfa66b8b4330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:24.428347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af49bc-a63c-434a-8d5b-cfa66b8b4330 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:24.434895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:04:24.451002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1141/2289] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:24 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:04:30.745963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd297a7-e6bd-4793-ad01-d1f1c0669863 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:30.746444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd297a7-e6bd-4793-ad01-d1f1c0669863 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:30.746543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd297a7-e6bd-4793-ad01-d1f1c0669863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:30.765519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-8bd297a7-e6bd-4793-ad01-d1f1c0669863 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:04:30.766796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8bd297a7-e6bd-4793-ad01-d1f1c0669863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:30.779266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:04:30.825543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1149/2290] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 05:04:30 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:04:30.837808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af49bc-a63c-434a-8d5b-cfa66b8b4330 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:30.903603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af49bc-a63c-434a-8d5b-cfa66b8b4330 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:30.908528 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dd72c-4311-4882-9d62-1b0ffc189681 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:30.909135 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dd72c-4311-4882-9d62-1b0ffc189681 service swift] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:30.909409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dd72c-4311-4882-9d62-1b0ffc189681 service swift] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:30.909673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dd72c-4311-4882-9d62-1b0ffc189681 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:30.919410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dd72c-4311-4882-9d62-1b0ffc189681 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:30.921099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4dd72c-4311-4882-9d62-1b0ffc189681 service swift] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:30.929514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1142/2291] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:04:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:04:35.566733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd297a7-e6bd-4793-ad01-d1f1c0669863 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:35.568166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8bd297a7-e6bd-4793-ad01-d1f1c0669863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:35.584751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd297a7-e6bd-4793-ad01-d1f1c0669863 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:35.590402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b40cf-ef7b-44a1-9dc6-556c18b7cb4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:35.595163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b40cf-ef7b-44a1-9dc6-556c18b7cb4d service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:35.595163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b40cf-ef7b-44a1-9dc6-556c18b7cb4d service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:35.595163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b40cf-ef7b-44a1-9dc6-556c18b7cb4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:35.595163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-770b40cf-ef7b-44a1-9dc6-556c18b7cb4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:35.780047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b40cf-ef7b-44a1-9dc6-556c18b7cb4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:35.781551 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b40cf-ef7b-44a1-9dc6-556c18b7cb4d service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:35.791572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1150/2292] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:35 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:04:40.926158 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=32 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:04:45.786771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=45 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:04:48.055010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dd72c-4311-4882-9d62-1b0ffc189681 service swift] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:48.064484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dd72c-4311-4882-9d62-1b0ffc189681 service swift] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:48.068142 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2daf2e-5de1-4552-934c-1cca64d54d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:48.071739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2daf2e-5de1-4552-934c-1cca64d54d9a service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:48.071739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2daf2e-5de1-4552-934c-1cca64d54d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:48.071739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2daf2e-5de1-4552-934c-1cca64d54d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:48.071739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3d2daf2e-5de1-4552-934c-1cca64d54d9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:48.081137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2daf2e-5de1-4552-934c-1cca64d54d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:48.082541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2daf2e-5de1-4552-934c-1cca64d54d9a service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:48.090563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1143/2293] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:48 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:04:48.897314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b40cf-ef7b-44a1-9dc6-556c18b7cb4d service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:48.899588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-770b40cf-ef7b-44a1-9dc6-556c18b7cb4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:48.986748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b40cf-ef7b-44a1-9dc6-556c18b7cb4d service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:48.992836 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7703b4-87c1-4c23-9018-8eef9ed53a18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:48.993380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7703b4-87c1-4c23-9018-8eef9ed53a18 service placement] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:48.993655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7703b4-87c1-4c23-9018-8eef9ed53a18 service placement] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:48.993922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7703b4-87c1-4c23-9018-8eef9ed53a18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:48.995700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9d7703b4-87c1-4c23-9018-8eef9ed53a18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:49.009365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7703b4-87c1-4c23-9018-8eef9ed53a18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:49.011491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7703b4-87c1-4c23-9018-8eef9ed53a18 service placement] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:49.026574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1151/2294] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:48 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:04:52.732704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2daf2e-5de1-4552-934c-1cca64d54d9a service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:52.742326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2daf2e-5de1-4552-934c-1cca64d54d9a service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:52.746134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3dc11-a413-43f1-8e89-75247a47af61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:52.746738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3dc11-a413-43f1-8e89-75247a47af61 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:52.747546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3dc11-a413-43f1-8e89-75247a47af61 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:52.751896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3dc11-a413-43f1-8e89-75247a47af61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:52.751896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-46f3dc11-a413-43f1-8e89-75247a47af61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:52.760074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3dc11-a413-43f1-8e89-75247a47af61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:52.761506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3dc11-a413-43f1-8e89-75247a47af61 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:52.769753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1144/2295] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:52 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:04:57.327780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7703b4-87c1-4c23-9018-8eef9ed53a18 service placement] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:04:57.340542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7703b4-87c1-4c23-9018-8eef9ed53a18 service placement] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:04:57.346339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00d682-cc95-49dd-8f8f-0dd19ffd63ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:04:57.347069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00d682-cc95-49dd-8f8f-0dd19ffd63ee service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:04:57.347368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00d682-cc95-49dd-8f8f-0dd19ffd63ee service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:04:57.347640 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00d682-cc95-49dd-8f8f-0dd19ffd63ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:04:57.349359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6c00d682-cc95-49dd-8f8f-0dd19ffd63ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:04:57.362293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00d682-cc95-49dd-8f8f-0dd19ffd63ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:04:57.363895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00d682-cc95-49dd-8f8f-0dd19ffd63ee service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:04:57.375813 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1152/2296] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:04:57 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:05:02.766084 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=12 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:05:06.716955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc9b73-6c17-412e-9e8d-ee26f53cdd07 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:05:06.717523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc9b73-6c17-412e-9e8d-ee26f53cdd07 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:05:06.717523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc9b73-6c17-412e-9e8d-ee26f53cdd07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:05:06.737246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-62fc9b73-6c17-412e-9e8d-ee26f53cdd07 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:05:06.738468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-62fc9b73-6c17-412e-9e8d-ee26f53cdd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:05:06.746601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1145/2297] 10.222.0.105 () {58 vars in 972 bytes} [Wed Aug 24 05:05:06 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:05:06.769074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00d682-cc95-49dd-8f8f-0dd19ffd63ee service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:05:06.780364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00d682-cc95-49dd-8f8f-0dd19ffd63ee service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:05:06.785497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6929b8d-969e-4e80-80c8-45e87315ff73 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:05:06.786086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929b8d-969e-4e80-80c8-45e87315ff73 service swift] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:05:06.786368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929b8d-969e-4e80-80c8-45e87315ff73 service swift] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:05:06.786637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6929b8d-969e-4e80-80c8-45e87315ff73 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:05:06.796240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6929b8d-969e-4e80-80c8-45e87315ff73 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:05:06.797825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6929b8d-969e-4e80-80c8-45e87315ff73 service swift] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:05:06.806680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1153/2298] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:05:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:05:13.563546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc9b73-6c17-412e-9e8d-ee26f53cdd07 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:05:13.576430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc9b73-6c17-412e-9e8d-ee26f53cdd07 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:05:13.581901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884bf69-4e47-488f-8287-c1b914459b4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:05:13.582501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884bf69-4e47-488f-8287-c1b914459b4b service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:05:13.582768 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884bf69-4e47-488f-8287-c1b914459b4b service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:05:13.583067 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884bf69-4e47-488f-8287-c1b914459b4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:05:13.584835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9884bf69-4e47-488f-8287-c1b914459b4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:05:13.644027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884bf69-4e47-488f-8287-c1b914459b4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:05:13.645602 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884bf69-4e47-488f-8287-c1b914459b4b service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:05:13.654890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1146/2299] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:05:13 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:05:16.802713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=44 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:05:18.340639 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6929b8d-969e-4e80-80c8-45e87315ff73 service swift] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:05:18.352706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6929b8d-969e-4e80-80c8-45e87315ff73 service swift] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:05:18.358132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa929b-869f-460e-b5bf-13d48a716629 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:05:18.358727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa929b-869f-460e-b5bf-13d48a716629 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:05:18.359004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa929b-869f-460e-b5bf-13d48a716629 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:05:18.359304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aa929b-869f-460e-b5bf-13d48a716629 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:05:18.360991 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-59aa929b-869f-460e-b5bf-13d48a716629 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:05:18.373389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa929b-869f-460e-b5bf-13d48a716629 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:05:18.375075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aa929b-869f-460e-b5bf-13d48a716629 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:05:18.386004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1154/2300] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:05:18 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:05:23.655250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=31 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:05:28.381080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=6 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:05:29.819988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884bf69-4e47-488f-8287-c1b914459b4b service nova] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:05:29.829108 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884bf69-4e47-488f-8287-c1b914459b4b service nova] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:05:29.832711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6526579-6f3c-4d4d-ae2d-6964ef1582fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:05:29.833299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6526579-6f3c-4d4d-ae2d-6964ef1582fb service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:05:29.833576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6526579-6f3c-4d4d-ae2d-6964ef1582fb service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:05:29.833843 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6526579-6f3c-4d4d-ae2d-6964ef1582fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:05:29.843388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6526579-6f3c-4d4d-ae2d-6964ef1582fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:05:29.844788 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6526579-6f3c-4d4d-ae2d-6964ef1582fb service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:05:29.854580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1147/2301] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:05:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:05:32.021141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa929b-869f-460e-b5bf-13d48a716629 service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:05:32.033368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aa929b-869f-460e-b5bf-13d48a716629 service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:05:32.039088 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079ce3b-7128-45f9-b8da-12bbb698798d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:05:32.039975 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079ce3b-7128-45f9-b8da-12bbb698798d service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:05:32.040312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079ce3b-7128-45f9-b8da-12bbb698798d service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:05:32.040650 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079ce3b-7128-45f9-b8da-12bbb698798d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:05:32.042445 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a079ce3b-7128-45f9-b8da-12bbb698798d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:05:32.054912 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079ce3b-7128-45f9-b8da-12bbb698798d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:05:32.057104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079ce3b-7128-45f9-b8da-12bbb698798d service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:05:32.067642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1155/2302] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:05:32 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:05:39.849880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=6 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:05:42.063858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=6 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:05:44.099760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6526579-6f3c-4d4d-ae2d-6964ef1582fb service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:05:44.111822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6526579-6f3c-4d4d-ae2d-6964ef1582fb service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:05:44.117268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90b613-0f68-4331-ae63-b60f13df60cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:05:44.117862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90b613-0f68-4331-ae63-b60f13df60cd service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:05:44.118141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90b613-0f68-4331-ae63-b60f13df60cd service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:05:44.118415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90b613-0f68-4331-ae63-b60f13df60cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:05:44.120134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ce90b613-0f68-4331-ae63-b60f13df60cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:05:44.178784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90b613-0f68-4331-ae63-b60f13df60cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:05:44.180403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90b613-0f68-4331-ae63-b60f13df60cd service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:05:44.189762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1148/2303] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:05:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:05:54.185514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=28 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:05:56.521573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079ce3b-7128-45f9-b8da-12bbb698798d service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:05:56.533830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079ce3b-7128-45f9-b8da-12bbb698798d service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:05:56.539440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8af78-0e58-4e9f-ad64-3dc3a84fc145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:05:56.540042 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8af78-0e58-4e9f-ad64-3dc3a84fc145 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:05:56.540316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8af78-0e58-4e9f-ad64-3dc3a84fc145 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:05:56.540583 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d8af78-0e58-4e9f-ad64-3dc3a84fc145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:05:56.542309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-99d8af78-0e58-4e9f-ad64-3dc3a84fc145 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:05:56.609413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8af78-0e58-4e9f-ad64-3dc3a84fc145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:05:56.611092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d8af78-0e58-4e9f-ad64-3dc3a84fc145 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:05:56.620343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1156/2304] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:05:56 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:06:01.608828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bc6cf-ac49-4c45-8a60-98028e73cd4a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:06:01.609609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bc6cf-ac49-4c45-8a60-98028e73cd4a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:06:01.610074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bc6cf-ac49-4c45-8a60-98028e73cd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:06:01.643455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8c1bc6cf-ac49-4c45-8a60-98028e73cd4a None None] MFA Rules not processed for user `51926dd0d6934aed907806a5cb14dc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:06:01.645844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8c1bc6cf-ac49-4c45-8a60-98028e73cd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:06:01.656418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1149/2305] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:06:01 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:06:06.616692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=28 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:06:08.754502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8af78-0e58-4e9f-ad64-3dc3a84fc145 service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:06:08.756197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-99d8af78-0e58-4e9f-ad64-3dc3a84fc145 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:06:08.836601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d8af78-0e58-4e9f-ad64-3dc3a84fc145 service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:06:08.841370 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:06:08.842017 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:06:08.842291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:06:08.842562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] PATH_INFO: `/v3/users/3cc9d727a1e044a397b8b8b9883d5dcb` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:06:08.843739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc9d727a1e044a397b8b8b9883d5dcb)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:06:08.845702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:06:09.114039 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc9d727a1e044a397b8b8b9883d5dcb'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:06:09.119743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc9d727a1e044a397b8b8b9883d5dcb'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:06:09.119743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc9d727a1e044a397b8b8b9883d5dcb'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:06:09.197218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc9d727a1e044a397b8b8b9883d5dcb'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:06:09.212196 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4cb7f7df-e3cc-455c-be11-931a2d6298f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc9d727a1e044a397b8b8b9883d5dcb'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:06:09.225789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1157/2306] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:06:08 2022] DELETE /identity/v3/users/3cc9d727a1e044a397b8b8b9883d5dcb => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:06:09.479199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bc6cf-ac49-4c45-8a60-98028e73cd4a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:06:09.493724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bc6cf-ac49-4c45-8a60-98028e73cd4a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:06:09.597244 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:06:09.597898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:06:09.598174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:06:09.598460 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] PATH_INFO: `/v3/projects/2dd4d1addbd348c0a9608c7b3fe27157` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:06:09.599670 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd4d1addbd348c0a9608c7b3fe27157)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:06:09.601684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:06:09.887113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd4d1addbd348c0a9608c7b3fe27157', 'request_id': 'req-14262dca-6d81-4030-8bc2-98d99947eb85'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:06:09.895242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd4d1addbd348c0a9608c7b3fe27157', 'request_id': 'req-14262dca-6d81-4030-8bc2-98d99947eb85'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:06:09.998971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd4d1addbd348c0a9608c7b3fe27157', 'request_id': 'req-14262dca-6d81-4030-8bc2-98d99947eb85'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:06:10.012139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] The token cache is being invalidate because project 2dd4d1addbd348c0a9608c7b3fe27157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:06:10.012534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:06:10.013213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:06:10.014972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1150/2307] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:06:09 2022] DELETE /identity/v3/projects/2dd4d1addbd348c0a9608c7b3fe27157 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:06:15.048086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a39da-bf03-45d0-a394-813bb0901630 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:06:15.048650 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a39da-bf03-45d0-a394-813bb0901630 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:06:15.048650 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a39da-bf03-45d0-a394-813bb0901630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:06:15.071216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-6e9a39da-bf03-45d0-a394-813bb0901630 None None] MFA Rules not processed for user `e45111c49aa74f76874d3a0c747034ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:06:15.072437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6e9a39da-bf03-45d0-a394-813bb0901630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:06:15.131588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1158/2308] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:06:15 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:06:15.450367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:06:15.451829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:06:15.509448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14262dca-6d81-4030-8bc2-98d99947eb85 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:06:15.514051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3625b7c-e39f-426a-820c-4c5cbb294ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:06:15.514670 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3625b7c-e39f-426a-820c-4c5cbb294ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:06:15.514945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3625b7c-e39f-426a-820c-4c5cbb294ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:06:15.522272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3625b7c-e39f-426a-820c-4c5cbb294ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:06:15.522272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d3625b7c-e39f-426a-820c-4c5cbb294ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:06:15.527604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3625b7c-e39f-426a-820c-4c5cbb294ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:06:15.528976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3625b7c-e39f-426a-820c-4c5cbb294ba5 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:06:15.536824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1151/2309] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:06:15 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:06:25.124544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:06:25.533391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:06:55.104015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25b6e-28c2-4ee3-938c-a4398b0aa011 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:06:55.104556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25b6e-28c2-4ee3-938c-a4398b0aa011 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:06:55.104556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa25b6e-28c2-4ee3-938c-a4398b0aa011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:06:55.124458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-0fa25b6e-28c2-4ee3-938c-a4398b0aa011 None None] MFA Rules not processed for user `842e7dea2bb24389b900d29351a0bd33`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:06:55.125650 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0fa25b6e-28c2-4ee3-938c-a4398b0aa011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:06:55.182786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1159/2310] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:06:55 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:07:05.174968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=25 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:07:30.945917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3625b7c-e39f-426a-820c-4c5cbb294ba5 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:30.947481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d3625b7c-e39f-426a-820c-4c5cbb294ba5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:31.102438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3625b7c-e39f-426a-820c-4c5cbb294ba5 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:31.110747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e1f40-7d71-4ec7-8b0c-23d016516d84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:31.110747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e1f40-7d71-4ec7-8b0c-23d016516d84 service placement] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:31.110747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e1f40-7d71-4ec7-8b0c-23d016516d84 service placement] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:31.110747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e1f40-7d71-4ec7-8b0c-23d016516d84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:31.112143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2c1e1f40-7d71-4ec7-8b0c-23d016516d84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:31.125197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e1f40-7d71-4ec7-8b0c-23d016516d84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:31.127076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e1f40-7d71-4ec7-8b0c-23d016516d84 service placement] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:31.139872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1152/2311] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:07:30 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:07:35.250489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2be5c7-116c-4dfe-a841-4ad7716f62f1 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:35.251371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2be5c7-116c-4dfe-a841-4ad7716f62f1 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:35.251371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2be5c7-116c-4dfe-a841-4ad7716f62f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:35.273498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-eb2be5c7-116c-4dfe-a841-4ad7716f62f1 None None] MFA Rules not processed for user `b11da4dac5804252b7310be096ae5d53`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:07:35.276002 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-eb2be5c7-116c-4dfe-a841-4ad7716f62f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:35.356958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1160/2312] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:07:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:07:41.135597 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=63 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:07:45.346606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=25 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:07:47.496216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e1f40-7d71-4ec7-8b0c-23d016516d84 service placement] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:47.498479 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2c1e1f40-7d71-4ec7-8b0c-23d016516d84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:47.535463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e1f40-7d71-4ec7-8b0c-23d016516d84 service placement] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:47.539357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8783e2-5ce2-48fa-9798-19c374781312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:47.539971 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8783e2-5ce2-48fa-9798-19c374781312 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:47.540251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8783e2-5ce2-48fa-9798-19c374781312 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:47.540521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8783e2-5ce2-48fa-9798-19c374781312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:47.542920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0a8783e2-5ce2-48fa-9798-19c374781312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:47.632100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8783e2-5ce2-48fa-9798-19c374781312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:47.633521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8783e2-5ce2-48fa-9798-19c374781312 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:47.652978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1153/2313] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:07:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:07:51.508180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2be5c7-116c-4dfe-a841-4ad7716f62f1 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:51.515210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-eb2be5c7-116c-4dfe-a841-4ad7716f62f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:51.526500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2be5c7-116c-4dfe-a841-4ad7716f62f1 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:51.532677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f98ac-aaaa-46a2-8af0-97890c8f84bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:51.533357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f98ac-aaaa-46a2-8af0-97890c8f84bd service placement] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:51.533644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f98ac-aaaa-46a2-8af0-97890c8f84bd service placement] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:51.533914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f98ac-aaaa-46a2-8af0-97890c8f84bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:51.535725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a23f98ac-aaaa-46a2-8af0-97890c8f84bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:51.549522 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f98ac-aaaa-46a2-8af0-97890c8f84bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:51.551535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f98ac-aaaa-46a2-8af0-97890c8f84bd service placement] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:51.561999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1161/2314] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:07:51 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:07:51.659881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8783e2-5ce2-48fa-9798-19c374781312 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:51.661575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0a8783e2-5ce2-48fa-9798-19c374781312 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:51.676361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8783e2-5ce2-48fa-9798-19c374781312 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:51.681804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:51.682506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:51.682886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:51.687451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] PATH_INFO: `/v3/users/51926dd0d6934aed907806a5cb14dc2f` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:51.690044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51926dd0d6934aed907806a5cb14dc2f)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:51.691091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:51.881805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51926dd0d6934aed907806a5cb14dc2f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:07:51.882396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51926dd0d6934aed907806a5cb14dc2f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:07:51.883167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51926dd0d6934aed907806a5cb14dc2f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:07:51.970607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51926dd0d6934aed907806a5cb14dc2f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:07:51.988388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b3041c2b-bfe3-469c-bae4-532d6b806667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51926dd0d6934aed907806a5cb14dc2f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:07:52.005867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1154/2315] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:07:51 2022] DELETE /identity/v3/users/51926dd0d6934aed907806a5cb14dc2f => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:07:52.309529 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f98ac-aaaa-46a2-8af0-97890c8f84bd service placement] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:52.323187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f98ac-aaaa-46a2-8af0-97890c8f84bd service placement] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:52.415814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:52.416467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:52.416745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:52.417117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] PATH_INFO: `/v3/projects/f2fd5d378aba473ab58753aecb74a065` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:52.418507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2fd5d378aba473ab58753aecb74a065)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:52.420743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:52.535621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2fd5d378aba473ab58753aecb74a065', 'request_id': 'req-d0fddd66-0015-4678-a09f-84b65fbaa85f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:07:52.543144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2fd5d378aba473ab58753aecb74a065', 'request_id': 'req-d0fddd66-0015-4678-a09f-84b65fbaa85f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:07:52.555312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2fd5d378aba473ab58753aecb74a065', 'request_id': 'req-d0fddd66-0015-4678-a09f-84b65fbaa85f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:07:52.651332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] The token cache is being invalidate because project f2fd5d378aba473ab58753aecb74a065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:07:52.652008 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:07:52.653060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d0fddd66-0015-4678-a09f-84b65fbaa85f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:07:52.658280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1162/2316] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:07:52 2022] DELETE /identity/v3/projects/f2fd5d378aba473ab58753aecb74a065 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:07:52.664934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a2a1f-9358-4ca8-8412-c2da662f48b1 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:52.664934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a2a1f-9358-4ca8-8412-c2da662f48b1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:52.664934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a2a1f-9358-4ca8-8412-c2da662f48b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:52.698426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:07:52.741865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5f1a2a1f-9358-4ca8-8412-c2da662f48b1 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:07:52.743142 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5f1a2a1f-9358-4ca8-8412-c2da662f48b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:52.819748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1155/2317] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:07:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:07:52.822215 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782be2b-ed17-49c5-9e83-173f427387b8 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:52.822710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782be2b-ed17-49c5-9e83-173f427387b8 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:52.823241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782be2b-ed17-49c5-9e83-173f427387b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:52.848987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-8782be2b-ed17-49c5-9e83-173f427387b8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:07:52.850388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8782be2b-ed17-49c5-9e83-173f427387b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:52.862769 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1163/2318] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:07:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:07:52.867302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a2a1f-9358-4ca8-8412-c2da662f48b1 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:52.884392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a2a1f-9358-4ca8-8412-c2da662f48b1 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:52.891952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ea79-8f17-49e1-9520-50ad3c3aa0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:52.892578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ea79-8f17-49e1-9520-50ad3c3aa0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:52.892852 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ea79-8f17-49e1-9520-50ad3c3aa0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:52.893122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ea79-8f17-49e1-9520-50ad3c3aa0d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:52.893778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636ea79-8f17-49e1-9520-50ad3c3aa0d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:07:52.894183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636ea79-8f17-49e1-9520-50ad3c3aa0d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:52.895881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636ea79-8f17-49e1-9520-50ad3c3aa0d4 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:52.916720 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1156/2319] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:07:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:07:52.918576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782be2b-ed17-49c5-9e83-173f427387b8 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:52.930548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782be2b-ed17-49c5-9e83-173f427387b8 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:52.936352 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dcd14-9300-4dcb-9f9c-590bc8a5c48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:52.936962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dcd14-9300-4dcb-9f9c-590bc8a5c48c admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:52.937241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dcd14-9300-4dcb-9f9c-590bc8a5c48c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:52.937533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3dcd14-9300-4dcb-9f9c-590bc8a5c48c admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:52.938132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3dcd14-9300-4dcb-9f9c-590bc8a5c48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:52.940810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3dcd14-9300-4dcb-9f9c-590bc8a5c48c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:53.081739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1164/2320] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:07:52 2022] POST /identity/v3/projects => generated 368 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:07:53.085517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ea79-8f17-49e1-9520-50ad3c3aa0d4 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:53.096040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ea79-8f17-49e1-9520-50ad3c3aa0d4 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:53.159399 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb918cfe-dd9e-4df3-b2b1-e6b44e68055b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:53.160220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb918cfe-dd9e-4df3-b2b1-e6b44e68055b admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:53.160518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb918cfe-dd9e-4df3-b2b1-e6b44e68055b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:53.160619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb918cfe-dd9e-4df3-b2b1-e6b44e68055b admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:53.161224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb918cfe-dd9e-4df3-b2b1-e6b44e68055b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:53.163320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb918cfe-dd9e-4df3-b2b1-e6b44e68055b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:53.282757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1157/2321] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:07:53 2022] POST /identity/v3/users => generated 285 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:07:53.286523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dcd14-9300-4dcb-9f9c-590bc8a5c48c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:53.297997 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3dcd14-9300-4dcb-9f9c-590bc8a5c48c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:53.303221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cf843-0086-43ff-81a1-6b2f5002c000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:53.303829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cf843-0086-43ff-81a1-6b2f5002c000 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:53.304106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cf843-0086-43ff-81a1-6b2f5002c000 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:53.304374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cf843-0086-43ff-81a1-6b2f5002c000 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:53.305012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cf843-0086-43ff-81a1-6b2f5002c000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:53.306633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cf843-0086-43ff-81a1-6b2f5002c000 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:53.317274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1165/2322] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:07:53 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:07:53.321292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb918cfe-dd9e-4df3-b2b1-e6b44e68055b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:53.332123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb918cfe-dd9e-4df3-b2b1-e6b44e68055b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:53.337031 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96339f9d-3d2c-4778-a81c-9c4a9958e850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:53.337638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339f9d-3d2c-4778-a81c-9c4a9958e850 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:53.337919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339f9d-3d2c-4778-a81c-9c4a9958e850 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:53.338215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339f9d-3d2c-4778-a81c-9c4a9958e850 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:53.338858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96339f9d-3d2c-4778-a81c-9c4a9958e850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:53.341145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96339f9d-3d2c-4778-a81c-9c4a9958e850 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:53.353618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1158/2323] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:07:53 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:07:53.358590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cf843-0086-43ff-81a1-6b2f5002c000 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:53.369631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cf843-0086-43ff-81a1-6b2f5002c000 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:53.374800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1b947-8077-454c-a1c2-60dd392e60cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:53.375531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1b947-8077-454c-a1c2-60dd392e60cf admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:53.375809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1b947-8077-454c-a1c2-60dd392e60cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:53.376078 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1b947-8077-454c-a1c2-60dd392e60cf admin admin] PATH_INFO: `/v3/projects/113d3966b26e48b8b3da11d091733106/users/5218295a3ddd4b07813705f94ff92c0c/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:53.392920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1b947-8077-454c-a1c2-60dd392e60cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=113d3966b26e48b8b3da11d091733106, user_id=5218295a3ddd4b07813705f94ff92c0c, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:53.395669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1b947-8077-454c-a1c2-60dd392e60cf admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:53.475795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1166/2324] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:07:53 2022] PUT /identity/v3/projects/113d3966b26e48b8b3da11d091733106/users/5218295a3ddd4b07813705f94ff92c0c/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:07:53.488046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374094-8f50-47e7-9538-def6d274c387 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:53.488335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374094-8f50-47e7-9538-def6d274c387 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:53.488609 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374094-8f50-47e7-9538-def6d274c387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:53.519511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-81374094-8f50-47e7-9538-def6d274c387 None None] MFA Rules not processed for user `5218295a3ddd4b07813705f94ff92c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:07:53.520725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-81374094-8f50-47e7-9538-def6d274c387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:53.562825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:07:53.587096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1159/2325] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:07:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:07:53.599283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1b947-8077-454c-a1c2-60dd392e60cf admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:07:53.601621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-40e1b947-8077-454c-a1c2-60dd392e60cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:07:53.632893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:07:53.695881 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1b947-8077-454c-a1c2-60dd392e60cf admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:07:53.700285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b6a775-9ffc-4556-b3db-0260da6ef0c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:07:53.700941 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a775-9ffc-4556-b3db-0260da6ef0c0 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:07:53.701218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a775-9ffc-4556-b3db-0260da6ef0c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:07:53.701489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a775-9ffc-4556-b3db-0260da6ef0c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:07:53.709939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6a775-9ffc-4556-b3db-0260da6ef0c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:07:53.711426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b6a775-9ffc-4556-b3db-0260da6ef0c0 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:07:53.720762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1167/2326] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:07:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:03.675344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=8 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:03.812330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=29 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:04.624326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81374094-8f50-47e7-9538-def6d274c387 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:04.688922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81374094-8f50-47e7-9538-def6d274c387 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:04.693833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:04.694466 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:04.694769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:04.695092 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] PATH_INFO: `/v3/users/5218295a3ddd4b07813705f94ff92c0c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:04.696345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5218295a3ddd4b07813705f94ff92c0c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:04.698434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:04.921555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5218295a3ddd4b07813705f94ff92c0c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:04.922196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5218295a3ddd4b07813705f94ff92c0c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:04.922789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5218295a3ddd4b07813705f94ff92c0c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:04.983831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5218295a3ddd4b07813705f94ff92c0c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:04.997676 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5218295a3ddd4b07813705f94ff92c0c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:05.009770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1160/2327] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:08:04 2022] DELETE /identity/v3/users/5218295a3ddd4b07813705f94ff92c0c => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:05.015548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a8ca4-1c57-4edc-9ff9-da37a5deb262 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:05.015548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a8ca4-1c57-4edc-9ff9-da37a5deb262 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:05.016107 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a8ca4-1c57-4edc-9ff9-da37a5deb262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:05.035805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b86a8ca4-1c57-4edc-9ff9-da37a5deb262 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:05.037056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b86a8ca4-1c57-4edc-9ff9-da37a5deb262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:05.100796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1168/2328] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:05.110922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:05.112450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:05.169603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a46a23-b8fc-4d6c-995b-b258fe8f319d admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:05.173566 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc871fa3-e6d4-4740-96a8-ba925d72513a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:05.174179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc871fa3-e6d4-4740-96a8-ba925d72513a service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:05.174463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc871fa3-e6d4-4740-96a8-ba925d72513a service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:05.174729 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc871fa3-e6d4-4740-96a8-ba925d72513a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:05.184992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc871fa3-e6d4-4740-96a8-ba925d72513a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:05.186475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc871fa3-e6d4-4740-96a8-ba925d72513a service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:05.197165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1161/2329] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:05.598141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a8ca4-1c57-4edc-9ff9-da37a5deb262 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:05.599882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b86a8ca4-1c57-4edc-9ff9-da37a5deb262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:05.612438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a8ca4-1c57-4edc-9ff9-da37a5deb262 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:05.617416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:05.618050 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:05.618327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:05.618594 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] PATH_INFO: `/v3/projects/113d3966b26e48b8b3da11d091733106` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:05.619764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=113d3966b26e48b8b3da11d091733106)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:05.621746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:05.756340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '113d3966b26e48b8b3da11d091733106', 'request_id': 'req-0a475a04-c7fe-46f1-b7a5-8883344658c6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:05.763333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '113d3966b26e48b8b3da11d091733106', 'request_id': 'req-0a475a04-c7fe-46f1-b7a5-8883344658c6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:05.769262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '113d3966b26e48b8b3da11d091733106', 'request_id': 'req-0a475a04-c7fe-46f1-b7a5-8883344658c6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:05.840123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] The token cache is being invalidate because project 113d3966b26e48b8b3da11d091733106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:08:05.840407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:05.841993 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0a475a04-c7fe-46f1-b7a5-8883344658c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:05.843598 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1169/2330] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:08:05 2022] DELETE /identity/v3/projects/113d3966b26e48b8b3da11d091733106 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:05.848613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a5f18-3c69-41ca-905d-96982f39abf0 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:05.849101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a5f18-3c69-41ca-905d-96982f39abf0 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:05.849213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120a5f18-3c69-41ca-905d-96982f39abf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:05.869146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-120a5f18-3c69-41ca-905d-96982f39abf0 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:05.870352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-120a5f18-3c69-41ca-905d-96982f39abf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:05.888564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:05.929565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1162/2331] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:05.938490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed0c1c-aba5-4a2b-b8df-a6e737fd2fa9 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:05.938799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed0c1c-aba5-4a2b-b8df-a6e737fd2fa9 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:05.939103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed0c1c-aba5-4a2b-b8df-a6e737fd2fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:05.959044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-6bed0c1c-aba5-4a2b-b8df-a6e737fd2fa9 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:05.960245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6bed0c1c-aba5-4a2b-b8df-a6e737fd2fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:05.968176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1170/2332] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:05.972505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a5f18-3c69-41ca-905d-96982f39abf0 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:05.982670 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120a5f18-3c69-41ca-905d-96982f39abf0 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:05.987438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb6eb-e87b-4445-b49e-a616b14b2a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:05.988219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb6eb-e87b-4445-b49e-a616b14b2a0c admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:05.988826 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb6eb-e87b-4445-b49e-a616b14b2a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:05.989097 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618eb6eb-e87b-4445-b49e-a616b14b2a0c admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:05.989733 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eb6eb-e87b-4445-b49e-a616b14b2a0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:08:05.990027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eb6eb-e87b-4445-b49e-a616b14b2a0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:05.991736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618eb6eb-e87b-4445-b49e-a616b14b2a0c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:06.004264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1163/2333] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:08:05 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:08:06.008575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed0c1c-aba5-4a2b-b8df-a6e737fd2fa9 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:06.018504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed0c1c-aba5-4a2b-b8df-a6e737fd2fa9 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:06.023131 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17f24-80fe-455d-b11a-824ddab1534f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:06.023726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17f24-80fe-455d-b11a-824ddab1534f admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:06.023999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17f24-80fe-455d-b11a-824ddab1534f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:06.024267 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17f24-80fe-455d-b11a-824ddab1534f admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:06.024889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e17f24-80fe-455d-b11a-824ddab1534f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:06.026749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e17f24-80fe-455d-b11a-824ddab1534f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:06.142056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1171/2334] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:08:06 2022] POST /identity/v3/projects => generated 400 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:06.146207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb6eb-e87b-4445-b49e-a616b14b2a0c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:06.157461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618eb6eb-e87b-4445-b49e-a616b14b2a0c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:06.224280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd78fc-bfa0-4b19-af73-44a037480b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:06.224894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd78fc-bfa0-4b19-af73-44a037480b75 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:06.225170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd78fc-bfa0-4b19-af73-44a037480b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:06.225445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd78fc-bfa0-4b19-af73-44a037480b75 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:06.226285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd78fc-bfa0-4b19-af73-44a037480b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:06.229817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd78fc-bfa0-4b19-af73-44a037480b75 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:06.306032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1164/2335] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:08:06 2022] POST /identity/v3/users => generated 301 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:06.311546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17f24-80fe-455d-b11a-824ddab1534f admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:06.323316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17f24-80fe-455d-b11a-824ddab1534f admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:06.328897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fb3ee-1813-43ae-a82e-b574796a539c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:06.329572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fb3ee-1813-43ae-a82e-b574796a539c admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:06.329857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fb3ee-1813-43ae-a82e-b574796a539c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:06.330124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2fb3ee-1813-43ae-a82e-b574796a539c admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:06.330877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2fb3ee-1813-43ae-a82e-b574796a539c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:06.332538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2fb3ee-1813-43ae-a82e-b574796a539c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:06.343961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1172/2336] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:06 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:06.352725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd78fc-bfa0-4b19-af73-44a037480b75 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:06.364144 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd78fc-bfa0-4b19-af73-44a037480b75 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:06.369056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb1f7d-51ab-415a-a654-154d47caf3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:06.369665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb1f7d-51ab-415a-a654-154d47caf3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:06.369937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb1f7d-51ab-415a-a654-154d47caf3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:06.370198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb1f7d-51ab-415a-a654-154d47caf3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:06.370844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb1f7d-51ab-415a-a654-154d47caf3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:06.373569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb1f7d-51ab-415a-a654-154d47caf3c8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:06.384916 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1165/2337] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:06 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:06.389481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fb3ee-1813-43ae-a82e-b574796a539c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:06.400206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2fb3ee-1813-43ae-a82e-b574796a539c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:06.407287 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3edcce-8e30-485b-bd25-2cd544b61200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:06.407976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3edcce-8e30-485b-bd25-2cd544b61200 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:06.408248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3edcce-8e30-485b-bd25-2cd544b61200 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:06.408512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3edcce-8e30-485b-bd25-2cd544b61200 admin admin] PATH_INFO: `/v3/projects/7a349277841b462d89f838f936077c78/users/f1672bc83ef34e51baf9eb3e27212d15/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:06.426032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3edcce-8e30-485b-bd25-2cd544b61200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a349277841b462d89f838f936077c78, user_id=f1672bc83ef34e51baf9eb3e27212d15, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:06.428786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3edcce-8e30-485b-bd25-2cd544b61200 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:06.531215 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1173/2338] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:06 2022] PUT /identity/v3/projects/7a349277841b462d89f838f936077c78/users/f1672bc83ef34e51baf9eb3e27212d15/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:06.536521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e74200-ede9-4a46-b877-9f24f593cf26 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:06.536664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e74200-ede9-4a46-b877-9f24f593cf26 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:06.536938 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e74200-ede9-4a46-b877-9f24f593cf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:06.557897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e5e74200-ede9-4a46-b877-9f24f593cf26 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:06.559877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e5e74200-ede9-4a46-b877-9f24f593cf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:06.627056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1166/2339] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:06.637100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3edcce-8e30-485b-bd25-2cd544b61200 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:06.639956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-7a3edcce-8e30-485b-bd25-2cd544b61200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:06.665565 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:06.707782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3edcce-8e30-485b-bd25-2cd544b61200 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:06.711541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ad702-92b2-41d0-b9d8-9476bf3485e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:06.712153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ad702-92b2-41d0-b9d8-9476bf3485e1 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:06.712431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ad702-92b2-41d0-b9d8-9476bf3485e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:06.712702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ad702-92b2-41d0-b9d8-9476bf3485e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:06.722726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ad702-92b2-41d0-b9d8-9476bf3485e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:06.724274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ad702-92b2-41d0-b9d8-9476bf3485e1 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:06.739055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1174/2340] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:08.168143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21233737-9f53-43c1-8513-d158b4c05432 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:08.168681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21233737-9f53-43c1-8513-d158b4c05432 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:08.174468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21233737-9f53-43c1-8513-d158b4c05432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:08.203086 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-21233737-9f53-43c1-8513-d158b4c05432 None None] MFA Rules not processed for user `f1672bc83ef34e51baf9eb3e27212d15`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:08.204396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-21233737-9f53-43c1-8513-d158b4c05432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:08.222647 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:08.276394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1167/2341] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:08 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:08.281893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ad702-92b2-41d0-b9d8-9476bf3485e1 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:08.291660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ad702-92b2-41d0-b9d8-9476bf3485e1 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:08.297224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c49375-99bf-4fef-8fcf-93f2a102d10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:08.297873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c49375-99bf-4fef-8fcf-93f2a102d10d admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:08.302349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c49375-99bf-4fef-8fcf-93f2a102d10d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:08.302349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c49375-99bf-4fef-8fcf-93f2a102d10d admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:08.302349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c49375-99bf-4fef-8fcf-93f2a102d10d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:08.302349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c49375-99bf-4fef-8fcf-93f2a102d10d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:08.397330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1175/2342] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:08:08 2022] POST /identity/v3/projects => generated 406 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:08.401549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21233737-9f53-43c1-8513-d158b4c05432 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:08.412943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21233737-9f53-43c1-8513-d158b4c05432 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:08.471560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26dd7d-4fcd-4a90-9fdd-5125b71907a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:08.472188 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26dd7d-4fcd-4a90-9fdd-5125b71907a5 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:08.472489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26dd7d-4fcd-4a90-9fdd-5125b71907a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:08.472800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26dd7d-4fcd-4a90-9fdd-5125b71907a5 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:08.473440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26dd7d-4fcd-4a90-9fdd-5125b71907a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:08.475560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26dd7d-4fcd-4a90-9fdd-5125b71907a5 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:08.560044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1168/2343] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:08:08 2022] POST /identity/v3/users => generated 310 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:08:08.564648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c49375-99bf-4fef-8fcf-93f2a102d10d admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:08.575739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c49375-99bf-4fef-8fcf-93f2a102d10d admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:08.580399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b711a-feb9-4959-87c8-94654fdb16a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:08.580998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b711a-feb9-4959-87c8-94654fdb16a3 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:08.581275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b711a-feb9-4959-87c8-94654fdb16a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:08.581546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b711a-feb9-4959-87c8-94654fdb16a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:08.582192 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b711a-feb9-4959-87c8-94654fdb16a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:08.583877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b711a-feb9-4959-87c8-94654fdb16a3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:08.594768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1176/2344] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:08 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:08.599082 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26dd7d-4fcd-4a90-9fdd-5125b71907a5 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:08.610093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26dd7d-4fcd-4a90-9fdd-5125b71907a5 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:08.614890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5aec5-71a2-4f96-ab3d-ef2dbacc195a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:08.615580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5aec5-71a2-4f96-ab3d-ef2dbacc195a admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:08.615858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5aec5-71a2-4f96-ab3d-ef2dbacc195a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:08.616130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5aec5-71a2-4f96-ab3d-ef2dbacc195a admin admin] PATH_INFO: `/v3/projects/9015e536e44647b088860b91c860f394/users/10da6a0aec6347928df5aa9f26949875/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:08.631752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5aec5-71a2-4f96-ab3d-ef2dbacc195a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9015e536e44647b088860b91c860f394, user_id=10da6a0aec6347928df5aa9f26949875, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:08.634386 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5aec5-71a2-4f96-ab3d-ef2dbacc195a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:08.708336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1169/2345] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:08 2022] PUT /identity/v3/projects/9015e536e44647b088860b91c860f394/users/10da6a0aec6347928df5aa9f26949875/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:08.712487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b711a-feb9-4959-87c8-94654fdb16a3 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:08.723784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b711a-feb9-4959-87c8-94654fdb16a3 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:08.783424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbb582-6a51-4f12-a87a-7cb1a50ffedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:08.784032 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbb582-6a51-4f12-a87a-7cb1a50ffedf admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:08.784309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbb582-6a51-4f12-a87a-7cb1a50ffedf admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:08.784577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbb582-6a51-4f12-a87a-7cb1a50ffedf admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:08.785204 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbb582-6a51-4f12-a87a-7cb1a50ffedf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:08.786782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bbb582-6a51-4f12-a87a-7cb1a50ffedf admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:08.798336 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1177/2346] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:08 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:08.802381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5aec5-71a2-4f96-ab3d-ef2dbacc195a admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:08.814304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5aec5-71a2-4f96-ab3d-ef2dbacc195a admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:08.819112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed442f-4175-41d6-a2d2-20ae73ecb0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:08.819776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed442f-4175-41d6-a2d2-20ae73ecb0eb admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:08.820048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed442f-4175-41d6-a2d2-20ae73ecb0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:08.820317 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed442f-4175-41d6-a2d2-20ae73ecb0eb admin admin] PATH_INFO: `/v3/projects/9015e536e44647b088860b91c860f394/users/10da6a0aec6347928df5aa9f26949875/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:08.822217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed442f-4175-41d6-a2d2-20ae73ecb0eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9015e536e44647b088860b91c860f394, user_id=10da6a0aec6347928df5aa9f26949875, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:08.824926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed442f-4175-41d6-a2d2-20ae73ecb0eb admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:08.835498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1170/2347] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:08 2022] PUT /identity/v3/projects/9015e536e44647b088860b91c860f394/users/10da6a0aec6347928df5aa9f26949875/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:10.297477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bac4a-5634-430b-bc5b-52d03b8ca365 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:10.297477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bac4a-5634-430b-bc5b-52d03b8ca365 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:10.297477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bac4a-5634-430b-bc5b-52d03b8ca365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:10.326823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f47bac4a-5634-430b-bc5b-52d03b8ca365 None None] MFA Rules not processed for user `10da6a0aec6347928df5aa9f26949875`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:10.328095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f47bac4a-5634-430b-bc5b-52d03b8ca365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:10.406173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1178/2348] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:10 2022] POST /identity/v3/auth/tokens => generated 3359 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:11.746289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fc5f6-232f-48b1-aadf-afe36922d940 None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:11.746867 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fc5f6-232f-48b1-aadf-afe36922d940 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:11.747664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51fc5f6-232f-48b1-aadf-afe36922d940 None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:11.751260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1171/2349] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 05:08:11 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 05:08:11.761823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b5107-5e5e-4641-9378-27d68917a638 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:11.761823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b5107-5e5e-4641-9378-27d68917a638 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:11.762614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b5107-5e5e-4641-9378-27d68917a638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:11.780539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:11.817737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ea7b5107-5e5e-4641-9378-27d68917a638 None None] MFA Rules not processed for user `ad8825c294534a7c88877b48b5f8f07f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:11.819248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ea7b5107-5e5e-4641-9378-27d68917a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:11.883768 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fc5f6-232f-48b1-aadf-afe36922d940 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:11.898144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1179/2350] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 05:08:11 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:11.916960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b5107-5e5e-4641-9378-27d68917a638 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:11.919822 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ea7b5107-5e5e-4641-9378-27d68917a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:11.936802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b5107-5e5e-4641-9378-27d68917a638 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:11.944080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60a4e7-a1b9-4609-8c5a-84a11b6333fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:11.944924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60a4e7-a1b9-4609-8c5a-84a11b6333fd service placement] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:11.945375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60a4e7-a1b9-4609-8c5a-84a11b6333fd service placement] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:11.945649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de60a4e7-a1b9-4609-8c5a-84a11b6333fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:11.956972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60a4e7-a1b9-4609-8c5a-84a11b6333fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:11.958707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de60a4e7-a1b9-4609-8c5a-84a11b6333fd service placement] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:11.970923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1180/2351] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:11 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:11.990160 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51fc5f6-232f-48b1-aadf-afe36922d940 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:11.995490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:11.996133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:11.996408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:11.996675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] PATH_INFO: `/v3/users/f1672bc83ef34e51baf9eb3e27212d15` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:11.998725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1672bc83ef34e51baf9eb3e27212d15)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:12.000856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:12.017669 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:12.192583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1672bc83ef34e51baf9eb3e27212d15'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:12.193613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1672bc83ef34e51baf9eb3e27212d15'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:12.194261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1672bc83ef34e51baf9eb3e27212d15'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:12.268470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1672bc83ef34e51baf9eb3e27212d15'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:12.282846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1672bc83ef34e51baf9eb3e27212d15'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:12.295676 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1172/2352] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:08:11 2022] DELETE /identity/v3/users/f1672bc83ef34e51baf9eb3e27212d15 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:08:12.636841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60a4e7-a1b9-4609-8c5a-84a11b6333fd service placement] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:12.650445 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de60a4e7-a1b9-4609-8c5a-84a11b6333fd service placement] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:12.742335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:12.743298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:12.743882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:12.744299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] PATH_INFO: `/v3/projects/7a349277841b462d89f838f936077c78` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:12.745490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a349277841b462d89f838f936077c78)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:12.748184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:12.897292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a349277841b462d89f838f936077c78', 'request_id': 'req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:12.905602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a349277841b462d89f838f936077c78', 'request_id': 'req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:12.912405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a349277841b462d89f838f936077c78', 'request_id': 'req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.007604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] The token cache is being invalidate because project 7a349277841b462d89f838f936077c78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:08:13.007604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.007990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.009885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1181/2353] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:08:12 2022] DELETE /identity/v3/projects/7a349277841b462d89f838f936077c78 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:13.014076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:13.015636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:13.030742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68b1c55-901a-4be9-8223-d7333165a257 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:13.108829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:13.109615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:13.109956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:13.110302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] PATH_INFO: `/v3/users/10da6a0aec6347928df5aa9f26949875` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:13.111606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] RBAC: Authorizing `identity:delete_user(user_id=10da6a0aec6347928df5aa9f26949875)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:13.113962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:13.316591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10da6a0aec6347928df5aa9f26949875'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.317324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10da6a0aec6347928df5aa9f26949875'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.319051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10da6a0aec6347928df5aa9f26949875'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.376351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10da6a0aec6347928df5aa9f26949875'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.395286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-0ae33ebc-6b21-4101-9b15-15794f4bb62c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10da6a0aec6347928df5aa9f26949875'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.408265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1173/2354] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:08:13 2022] DELETE /identity/v3/users/10da6a0aec6347928df5aa9f26949875 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:13.677942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:13.680550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:13.688381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:13.695729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd64ffa-3ee1-4892-9caa-3c45b4810fd5 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:13.763606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:13.764379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:13.764737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:13.765096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] PATH_INFO: `/v3/projects/9015e536e44647b088860b91c860f394` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:13.766293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9015e536e44647b088860b91c860f394)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:13.768319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:13.895690 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9015e536e44647b088860b91c860f394', 'request_id': 'req-0b293ba4-7773-4b8f-b83c-4a9220235d73'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.902493 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9015e536e44647b088860b91c860f394', 'request_id': 'req-0b293ba4-7773-4b8f-b83c-4a9220235d73'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.908602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9015e536e44647b088860b91c860f394', 'request_id': 'req-0b293ba4-7773-4b8f-b83c-4a9220235d73'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.955604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] The token cache is being invalidate because project 9015e536e44647b088860b91c860f394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:08:13.961105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.961105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0b293ba4-7773-4b8f-b83c-4a9220235d73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:13.961105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1182/2355] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:08:13 2022] DELETE /identity/v3/projects/9015e536e44647b088860b91c860f394 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:13.963082 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53553c-ba99-4005-8360-2ae3518074ff None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:13.963367 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53553c-ba99-4005-8360-2ae3518074ff None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:13.963638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53553c-ba99-4005-8360-2ae3518074ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:13.984762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8d53553c-ba99-4005-8360-2ae3518074ff None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:13.989423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8d53553c-ba99-4005-8360-2ae3518074ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:13.990160 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:14.046727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1174/2356] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:14.054421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32988dc-749f-4d81-9ff5-c56ef559cb1a None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:14.054709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32988dc-749f-4d81-9ff5-c56ef559cb1a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:14.054979 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32988dc-749f-4d81-9ff5-c56ef559cb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:14.075216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d32988dc-749f-4d81-9ff5-c56ef559cb1a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:14.076468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d32988dc-749f-4d81-9ff5-c56ef559cb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:14.084227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1183/2357] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:14 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:14.088368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53553c-ba99-4005-8360-2ae3518074ff None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:14.098600 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53553c-ba99-4005-8360-2ae3518074ff None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:14.103087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b2553-6424-4795-8cae-acc1ad72a417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:14.103685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b2553-6424-4795-8cae-acc1ad72a417 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:14.103965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b2553-6424-4795-8cae-acc1ad72a417 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:14.104232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b2553-6424-4795-8cae-acc1ad72a417 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:14.104874 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b2553-6424-4795-8cae-acc1ad72a417 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:08:14.105174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b2553-6424-4795-8cae-acc1ad72a417 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:14.106808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b2553-6424-4795-8cae-acc1ad72a417 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:14.118781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1175/2358] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:08:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:08:14.123629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32988dc-749f-4d81-9ff5-c56ef559cb1a None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:14.133047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32988dc-749f-4d81-9ff5-c56ef559cb1a None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:14.139457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b71446-d796-4706-9032-009817626fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:14.140056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b71446-d796-4706-9032-009817626fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:14.140332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b71446-d796-4706-9032-009817626fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:14.141800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b71446-d796-4706-9032-009817626fb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:14.141800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b71446-d796-4706-9032-009817626fb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:14.143151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b71446-d796-4706-9032-009817626fb4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:14.269707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1184/2359] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:08:14 2022] POST /identity/v3/projects => generated 382 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:14.274026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b2553-6424-4795-8cae-acc1ad72a417 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:14.288056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b2553-6424-4795-8cae-acc1ad72a417 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:14.355507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79001a99-55e2-4d39-9669-d678ba01c759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:14.355507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79001a99-55e2-4d39-9669-d678ba01c759 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:14.355507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79001a99-55e2-4d39-9669-d678ba01c759 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:14.355507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79001a99-55e2-4d39-9669-d678ba01c759 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:14.356072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79001a99-55e2-4d39-9669-d678ba01c759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:14.356072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79001a99-55e2-4d39-9669-d678ba01c759 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:14.430251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1176/2360] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:08:14 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:08:14.434502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b71446-d796-4706-9032-009817626fb4 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:14.444998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b71446-d796-4706-9032-009817626fb4 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:14.449586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d7267-37bf-424f-b68e-cb48598a4170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:14.456146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d7267-37bf-424f-b68e-cb48598a4170 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:14.456146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d7267-37bf-424f-b68e-cb48598a4170 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:14.456146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d7267-37bf-424f-b68e-cb48598a4170 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:14.456146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d7267-37bf-424f-b68e-cb48598a4170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:14.456146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d7267-37bf-424f-b68e-cb48598a4170 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:14.463191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1185/2361] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:14 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:14.469738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79001a99-55e2-4d39-9669-d678ba01c759 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:14.479866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79001a99-55e2-4d39-9669-d678ba01c759 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:14.485306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16327925-b65f-4a72-83c9-dac78d8b510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:14.485902 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16327925-b65f-4a72-83c9-dac78d8b510a admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:14.486179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16327925-b65f-4a72-83c9-dac78d8b510a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:14.486449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16327925-b65f-4a72-83c9-dac78d8b510a admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:14.487117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16327925-b65f-4a72-83c9-dac78d8b510a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:14.488704 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16327925-b65f-4a72-83c9-dac78d8b510a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:14.499860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1177/2362] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:14 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:14.510617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d7267-37bf-424f-b68e-cb48598a4170 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:14.518589 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d7267-37bf-424f-b68e-cb48598a4170 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:14.523493 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fc88a-bf85-4e8d-9987-137dbe94c08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:14.524155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fc88a-bf85-4e8d-9987-137dbe94c08f admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:14.524496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fc88a-bf85-4e8d-9987-137dbe94c08f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:14.524819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fc88a-bf85-4e8d-9987-137dbe94c08f admin admin] PATH_INFO: `/v3/projects/e9a6a739c691422592e32090a67f96ef/users/d51d96200d784d43a6205b63361f5b6d/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:14.551838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59fc88a-bf85-4e8d-9987-137dbe94c08f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a6a739c691422592e32090a67f96ef, user_id=d51d96200d784d43a6205b63361f5b6d, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:14.554492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59fc88a-bf85-4e8d-9987-137dbe94c08f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:14.608409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1186/2363] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:14 2022] PUT /identity/v3/projects/e9a6a739c691422592e32090a67f96ef/users/d51d96200d784d43a6205b63361f5b6d/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:14.619820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac647c92-0862-451c-9b38-91bde1859be7 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:14.620274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac647c92-0862-451c-9b38-91bde1859be7 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:14.620405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac647c92-0862-451c-9b38-91bde1859be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:14.650911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ac647c92-0862-451c-9b38-91bde1859be7 None None] MFA Rules not processed for user `d51d96200d784d43a6205b63361f5b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:14.652181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ac647c92-0862-451c-9b38-91bde1859be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:14.722464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1178/2364] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:14.727066 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fc88a-bf85-4e8d-9987-137dbe94c08f admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:14.761414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:14.795850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59fc88a-bf85-4e8d-9987-137dbe94c08f admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:14.800943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab9327-1f08-461c-bb10-8928f7770afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:14.801553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab9327-1f08-461c-bb10-8928f7770afb admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:14.801830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab9327-1f08-461c-bb10-8928f7770afb admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:14.802104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ab9327-1f08-461c-bb10-8928f7770afb admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:14.802740 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab9327-1f08-461c-bb10-8928f7770afb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:14.804668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ab9327-1f08-461c-bb10-8928f7770afb admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:14.898667 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1187/2365] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:08:14 2022] POST /identity/v3/projects => generated 382 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:14.899816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac647c92-0862-451c-9b38-91bde1859be7 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:14.906621 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:14.911290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac647c92-0862-451c-9b38-91bde1859be7 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:14.970826 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467428-921f-4069-8837-f70ce44a417c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:14.972802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467428-921f-4069-8837-f70ce44a417c admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:14.972802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467428-921f-4069-8837-f70ce44a417c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:14.972802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89467428-921f-4069-8837-f70ce44a417c admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:14.972802 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89467428-921f-4069-8837-f70ce44a417c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:14.974564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89467428-921f-4069-8837-f70ce44a417c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:15.075690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1179/2366] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:08:14 2022] POST /identity/v3/users => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:08:15.077691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab9327-1f08-461c-bb10-8928f7770afb admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:15.089679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ab9327-1f08-461c-bb10-8928f7770afb admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:15.095285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97645ae-46ef-451e-a192-b0563a72bd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:15.095906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97645ae-46ef-451e-a192-b0563a72bd63 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:15.096187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97645ae-46ef-451e-a192-b0563a72bd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:15.096459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97645ae-46ef-451e-a192-b0563a72bd63 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:15.097266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97645ae-46ef-451e-a192-b0563a72bd63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:15.098762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97645ae-46ef-451e-a192-b0563a72bd63 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:15.112851 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1188/2367] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:15 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:15.114533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467428-921f-4069-8837-f70ce44a417c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:15.129698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89467428-921f-4069-8837-f70ce44a417c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:15.134500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae21677-3d1e-4913-bf4a-47416bb1cf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:15.135260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae21677-3d1e-4913-bf4a-47416bb1cf09 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:15.135563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae21677-3d1e-4913-bf4a-47416bb1cf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:15.135831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae21677-3d1e-4913-bf4a-47416bb1cf09 admin admin] PATH_INFO: `/v3/projects/a35e038c01ab4420aa2ae78b4cb23059/users/ce938fa576b64da9b97b82faeb05433a/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:15.159702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae21677-3d1e-4913-bf4a-47416bb1cf09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a35e038c01ab4420aa2ae78b4cb23059, user_id=ce938fa576b64da9b97b82faeb05433a, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:15.162819 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae21677-3d1e-4913-bf4a-47416bb1cf09 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:15.248418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1180/2368] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:15 2022] PUT /identity/v3/projects/a35e038c01ab4420aa2ae78b4cb23059/users/ce938fa576b64da9b97b82faeb05433a/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:15.252501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97645ae-46ef-451e-a192-b0563a72bd63 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:15.263481 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97645ae-46ef-451e-a192-b0563a72bd63 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:15.332245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189ab69-fd92-4eb3-9c4f-8b1b092092fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:15.333269 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189ab69-fd92-4eb3-9c4f-8b1b092092fb admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:15.333714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189ab69-fd92-4eb3-9c4f-8b1b092092fb admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:15.334734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189ab69-fd92-4eb3-9c4f-8b1b092092fb admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:15.335634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189ab69-fd92-4eb3-9c4f-8b1b092092fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:15.337776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189ab69-fd92-4eb3-9c4f-8b1b092092fb admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:15.351669 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1189/2369] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:15 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:15.355781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae21677-3d1e-4913-bf4a-47416bb1cf09 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:15.366866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae21677-3d1e-4913-bf4a-47416bb1cf09 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:15.371598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e1bed-ea1c-47d5-8bbd-1f93474edd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:15.380071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e1bed-ea1c-47d5-8bbd-1f93474edd67 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:15.380071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e1bed-ea1c-47d5-8bbd-1f93474edd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:15.380071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e1bed-ea1c-47d5-8bbd-1f93474edd67 admin admin] PATH_INFO: `/v3/projects/a35e038c01ab4420aa2ae78b4cb23059/users/ce938fa576b64da9b97b82faeb05433a/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:15.380071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e1bed-ea1c-47d5-8bbd-1f93474edd67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a35e038c01ab4420aa2ae78b4cb23059, user_id=ce938fa576b64da9b97b82faeb05433a, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:15.380071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e1bed-ea1c-47d5-8bbd-1f93474edd67 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:15.390767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1181/2370] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:15 2022] PUT /identity/v3/projects/a35e038c01ab4420aa2ae78b4cb23059/users/ce938fa576b64da9b97b82faeb05433a/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:08:15.401257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ab806-04fa-40ac-82df-2060141ac967 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:15.401566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ab806-04fa-40ac-82df-2060141ac967 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:15.401840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ab806-04fa-40ac-82df-2060141ac967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:15.432955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-151ab806-04fa-40ac-82df-2060141ac967 None None] MFA Rules not processed for user `ce938fa576b64da9b97b82faeb05433a`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:15.435994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-151ab806-04fa-40ac-82df-2060141ac967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:15.507612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1190/2371] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:15 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:15.518622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e1bed-ea1c-47d5-8bbd-1f93474edd67 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:15.520453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f19e1bed-ea1c-47d5-8bbd-1f93474edd67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:15.578102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e1bed-ea1c-47d5-8bbd-1f93474edd67 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:15.584122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f92c9-f5ab-4b16-842f-4cc4c13c457a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:15.584962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f92c9-f5ab-4b16-842f-4cc4c13c457a service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:15.585258 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f92c9-f5ab-4b16-842f-4cc4c13c457a service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:15.590178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f92c9-f5ab-4b16-842f-4cc4c13c457a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:15.595517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f92c9-f5ab-4b16-842f-4cc4c13c457a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:15.596995 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f92c9-f5ab-4b16-842f-4cc4c13c457a service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:15.607643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1182/2372] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:15 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:15.711344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ab806-04fa-40ac-82df-2060141ac967 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:15.722238 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ab806-04fa-40ac-82df-2060141ac967 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:15.727475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7a44-1967-4f88-90e0-cd30d1fc0e11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:15.727475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7a44-1967-4f88-90e0-cd30d1fc0e11 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:15.727475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7a44-1967-4f88-90e0-cd30d1fc0e11 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:15.727475 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c7a44-1967-4f88-90e0-cd30d1fc0e11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:15.749750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:15.787653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c7a44-1967-4f88-90e0-cd30d1fc0e11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:15.789290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c7a44-1967-4f88-90e0-cd30d1fc0e11 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:15.800588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1191/2373] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:17.646958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b3138-34e9-419a-a9d5-21c16dc27ddc None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:17.647491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b3138-34e9-419a-a9d5-21c16dc27ddc None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:17.647591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b3138-34e9-419a-a9d5-21c16dc27ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:17.667233 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-030b3138-34e9-419a-a9d5-21c16dc27ddc None None] MFA Rules not processed for user `e45111c49aa74f76874d3a0c747034ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:17.673501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-030b3138-34e9-419a-a9d5-21c16dc27ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:17.725942 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1183/2374] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:17 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:18.905413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7a44-1967-4f88-90e0-cd30d1fc0e11 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:18.974156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c7a44-1967-4f88-90e0-cd30d1fc0e11 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:18.979251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:18.981160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:18.981160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:18.981160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] PATH_INFO: `/v3/users/d51d96200d784d43a6205b63361f5b6d` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:18.981615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d51d96200d784d43a6205b63361f5b6d)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:18.985124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:19.188823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd51d96200d784d43a6205b63361f5b6d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:19.189507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd51d96200d784d43a6205b63361f5b6d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:19.190118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd51d96200d784d43a6205b63361f5b6d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:19.311189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd51d96200d784d43a6205b63361f5b6d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:19.321343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd51d96200d784d43a6205b63361f5b6d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:19.335898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1192/2375] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:08:18 2022] DELETE /identity/v3/users/d51d96200d784d43a6205b63361f5b6d => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:19.340394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666904b-3ed8-4116-be6d-1063dfa6205a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:19.340979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666904b-3ed8-4116-be6d-1063dfa6205a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:19.341211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666904b-3ed8-4116-be6d-1063dfa6205a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:19.350961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:19.362286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6666904b-3ed8-4116-be6d-1063dfa6205a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:19.363530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6666904b-3ed8-4116-be6d-1063dfa6205a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:19.444679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1184/2376] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:19.455853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:19.457477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:19.515304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:19.521485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3519b7-22e5-4dc5-a279-8462a5ca0726 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:19.525596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3151726-ebc6-47f8-8e77-a98080339940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:19.527288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151726-ebc6-47f8-8e77-a98080339940 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:19.527846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151726-ebc6-47f8-8e77-a98080339940 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:19.529562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3151726-ebc6-47f8-8e77-a98080339940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:19.545133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3151726-ebc6-47f8-8e77-a98080339940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:19.547054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3151726-ebc6-47f8-8e77-a98080339940 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:19.558218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1193/2377] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:20.029883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666904b-3ed8-4116-be6d-1063dfa6205a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:20.031573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6666904b-3ed8-4116-be6d-1063dfa6205a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:20.043695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666904b-3ed8-4116-be6d-1063dfa6205a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:20.050171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:20.050826 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:20.051122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:20.051394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] PATH_INFO: `/v3/projects/e9a6a739c691422592e32090a67f96ef` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:20.052564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9a6a739c691422592e32090a67f96ef)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:20.055556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:20.160357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9a6a739c691422592e32090a67f96ef', 'request_id': 'req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:20.167201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9a6a739c691422592e32090a67f96ef', 'request_id': 'req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:20.270023 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9a6a739c691422592e32090a67f96ef', 'request_id': 'req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:20.282498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] The token cache is being invalidate because project e9a6a739c691422592e32090a67f96ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:08:20.282987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:20.284056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:20.286559 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1185/2378] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:08:20 2022] DELETE /identity/v3/projects/e9a6a739c691422592e32090a67f96ef => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:20.292820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3151726-ebc6-47f8-8e77-a98080339940 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:20.311755 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3151726-ebc6-47f8-8e77-a98080339940 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:20.384499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:20.385385 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:20.385670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:20.385943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] PATH_INFO: `/v3/users/ce938fa576b64da9b97b82faeb05433a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:20.387642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce938fa576b64da9b97b82faeb05433a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:20.389685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:20.607753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce938fa576b64da9b97b82faeb05433a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:20.608651 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce938fa576b64da9b97b82faeb05433a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:20.609084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce938fa576b64da9b97b82faeb05433a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:20.673798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce938fa576b64da9b97b82faeb05433a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:20.694843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-cd803882-8650-401d-a2ea-d994b9c9d856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce938fa576b64da9b97b82faeb05433a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:20.715231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1194/2379] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:08:20 2022] DELETE /identity/v3/users/ce938fa576b64da9b97b82faeb05433a => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:21.344481 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:21.346167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:21.361935 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf86c7-5eb6-41bf-a071-a5334dce15e3 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:21.427491 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:21.428143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:21.428426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:21.428685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] PATH_INFO: `/v3/projects/a35e038c01ab4420aa2ae78b4cb23059` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:21.430226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a35e038c01ab4420aa2ae78b4cb23059)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:21.432346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:21.534319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:21.542681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a35e038c01ab4420aa2ae78b4cb23059', 'request_id': 'req-189f081c-556a-4292-8af7-39d33ce740e4'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:21.549962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a35e038c01ab4420aa2ae78b4cb23059', 'request_id': 'req-189f081c-556a-4292-8af7-39d33ce740e4'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:21.641395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a35e038c01ab4420aa2ae78b4cb23059', 'request_id': 'req-189f081c-556a-4292-8af7-39d33ce740e4'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:21.653472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] The token cache is being invalidate because project a35e038c01ab4420aa2ae78b4cb23059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:08:21.653780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:21.654478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-189f081c-556a-4292-8af7-39d33ce740e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:21.656328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1186/2380] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:08:21 2022] DELETE /identity/v3/projects/a35e038c01ab4420aa2ae78b4cb23059 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:21.661228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19663345-bec4-4cd1-b0dc-f4e194ccf077 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:21.661765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19663345-bec4-4cd1-b0dc-f4e194ccf077 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:21.661879 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19663345-bec4-4cd1-b0dc-f4e194ccf077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:21.683235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-19663345-bec4-4cd1-b0dc-f4e194ccf077 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:21.684390 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-19663345-bec4-4cd1-b0dc-f4e194ccf077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:21.756186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1195/2381] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:21.764114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebc7d5-779f-41f7-941a-7b55d0a5a66d None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:21.764489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebc7d5-779f-41f7-941a-7b55d0a5a66d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:21.765833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ebc7d5-779f-41f7-941a-7b55d0a5a66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:21.793147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-d6ebc7d5-779f-41f7-941a-7b55d0a5a66d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:21.793697 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d6ebc7d5-779f-41f7-941a-7b55d0a5a66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:21.807917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1187/2382] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:21.809282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19663345-bec4-4cd1-b0dc-f4e194ccf077 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:21.816542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:21.820902 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19663345-bec4-4cd1-b0dc-f4e194ccf077 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:21.826914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbde10f-9f5d-4f5f-bc03-7b69dd0b95f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:21.833361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbde10f-9f5d-4f5f-bc03-7b69dd0b95f2 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:21.833361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbde10f-9f5d-4f5f-bc03-7b69dd0b95f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:21.833361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbde10f-9f5d-4f5f-bc03-7b69dd0b95f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:21.833361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbde10f-9f5d-4f5f-bc03-7b69dd0b95f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:08:21.833361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbde10f-9f5d-4f5f-bc03-7b69dd0b95f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:21.833361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbde10f-9f5d-4f5f-bc03-7b69dd0b95f2 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:21.849601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1196/2383] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:08:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:08:21.853789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebc7d5-779f-41f7-941a-7b55d0a5a66d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:21.868978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ebc7d5-779f-41f7-941a-7b55d0a5a66d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:21.885183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6704579-425b-4b6a-bc9c-530b8ae324d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:21.886148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6704579-425b-4b6a-bc9c-530b8ae324d4 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:21.886435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6704579-425b-4b6a-bc9c-530b8ae324d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:21.886709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6704579-425b-4b6a-bc9c-530b8ae324d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:21.887519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6704579-425b-4b6a-bc9c-530b8ae324d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:21.889417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6704579-425b-4b6a-bc9c-530b8ae324d4 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:22.007505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1188/2384] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:08:21 2022] POST /identity/v3/projects => generated 382 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:22.008664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbde10f-9f5d-4f5f-bc03-7b69dd0b95f2 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:22.019214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbde10f-9f5d-4f5f-bc03-7b69dd0b95f2 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:22.085845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1488bf-0e07-4e6d-858a-06e5cc4d4ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:22.087228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1488bf-0e07-4e6d-858a-06e5cc4d4ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:22.087506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1488bf-0e07-4e6d-858a-06e5cc4d4ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:22.087779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1488bf-0e07-4e6d-858a-06e5cc4d4ef1 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:22.088448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1488bf-0e07-4e6d-858a-06e5cc4d4ef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:22.090462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1488bf-0e07-4e6d-858a-06e5cc4d4ef1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:22.213660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1197/2385] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:08:22 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:08:22.218599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6704579-425b-4b6a-bc9c-530b8ae324d4 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:22.230403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6704579-425b-4b6a-bc9c-530b8ae324d4 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:22.236052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7e3b9-8778-4fff-8a0e-448329764933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:22.238547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7e3b9-8778-4fff-8a0e-448329764933 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:22.238913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7e3b9-8778-4fff-8a0e-448329764933 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:22.239247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb7e3b9-8778-4fff-8a0e-448329764933 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:22.239912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb7e3b9-8778-4fff-8a0e-448329764933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:22.241515 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb7e3b9-8778-4fff-8a0e-448329764933 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:22.255671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1189/2386] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:22 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:22.260254 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1488bf-0e07-4e6d-858a-06e5cc4d4ef1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:22.271325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1488bf-0e07-4e6d-858a-06e5cc4d4ef1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:22.276371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628c531-e4f9-442b-812e-d57f6696d222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:22.277005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628c531-e4f9-442b-812e-d57f6696d222 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:22.277284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628c531-e4f9-442b-812e-d57f6696d222 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:22.277550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628c531-e4f9-442b-812e-d57f6696d222 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:22.278184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628c531-e4f9-442b-812e-d57f6696d222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:22.279916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628c531-e4f9-442b-812e-d57f6696d222 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:22.292091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1198/2387] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:22 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:22.297469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7e3b9-8778-4fff-8a0e-448329764933 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:22.308482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb7e3b9-8778-4fff-8a0e-448329764933 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:22.314496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea3948-85fc-4668-8af9-e5d70fac2583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:22.315149 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea3948-85fc-4668-8af9-e5d70fac2583 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:22.315846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea3948-85fc-4668-8af9-e5d70fac2583 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:22.315846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ea3948-85fc-4668-8af9-e5d70fac2583 admin admin] PATH_INFO: `/v3/projects/9fbd8722fc4b4ca6a60dfb6983906644/users/ee066ff2a0b4492993bb7ca8a900e571/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:22.333601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea3948-85fc-4668-8af9-e5d70fac2583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fbd8722fc4b4ca6a60dfb6983906644, user_id=ee066ff2a0b4492993bb7ca8a900e571, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:22.336452 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ea3948-85fc-4668-8af9-e5d70fac2583 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:22.419451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1190/2388] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:22 2022] PUT /identity/v3/projects/9fbd8722fc4b4ca6a60dfb6983906644/users/ee066ff2a0b4492993bb7ca8a900e571/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:22.431838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9adbe6-f9a4-4721-9e31-bf636c7002ba None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:22.432207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9adbe6-f9a4-4721-9e31-bf636c7002ba None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:22.432483 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9adbe6-f9a4-4721-9e31-bf636c7002ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:22.461295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-8d9adbe6-f9a4-4721-9e31-bf636c7002ba None None] MFA Rules not processed for user `ee066ff2a0b4492993bb7ca8a900e571`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:22.462604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8d9adbe6-f9a4-4721-9e31-bf636c7002ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:22.540975 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1199/2389] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:08:22.545303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea3948-85fc-4668-8af9-e5d70fac2583 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:22.610011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ea3948-85fc-4668-8af9-e5d70fac2583 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:22.614791 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12006dab-6b4f-4628-a272-ed11597945bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:22.615453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12006dab-6b4f-4628-a272-ed11597945bb admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:22.615733 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12006dab-6b4f-4628-a272-ed11597945bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:22.616038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12006dab-6b4f-4628-a272-ed11597945bb admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:22.616673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12006dab-6b4f-4628-a272-ed11597945bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:22.618578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12006dab-6b4f-4628-a272-ed11597945bb admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:22.741850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1191/2390] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:08:22 2022] POST /identity/v3/projects => generated 382 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:22.745971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9adbe6-f9a4-4721-9e31-bf636c7002ba None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:22.757006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9adbe6-f9a4-4721-9e31-bf636c7002ba None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:22.821092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56fc36-a5c6-40ac-8366-e62ad34d2ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:22.821709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56fc36-a5c6-40ac-8366-e62ad34d2ace admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:22.821991 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56fc36-a5c6-40ac-8366-e62ad34d2ace admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:22.822261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56fc36-a5c6-40ac-8366-e62ad34d2ace admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:22.822987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56fc36-a5c6-40ac-8366-e62ad34d2ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:22.825733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56fc36-a5c6-40ac-8366-e62ad34d2ace admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:22.835780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:22.932508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1200/2391] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:08:22 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:08:22.936360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12006dab-6b4f-4628-a272-ed11597945bb admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:22.947574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12006dab-6b4f-4628-a272-ed11597945bb admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:22.960547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a99e5-e3f5-479f-93ed-b3ef19784e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:22.961279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a99e5-e3f5-479f-93ed-b3ef19784e32 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:22.961504 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a99e5-e3f5-479f-93ed-b3ef19784e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:22.961769 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a99e5-e3f5-479f-93ed-b3ef19784e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:22.962450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a99e5-e3f5-479f-93ed-b3ef19784e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:22.964312 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682a99e5-e3f5-479f-93ed-b3ef19784e32 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:22.977119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1192/2392] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:22 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:22.986898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56fc36-a5c6-40ac-8366-e62ad34d2ace admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:22.999592 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56fc36-a5c6-40ac-8366-e62ad34d2ace admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:23.004958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afca65-1778-4d03-963b-5b4470aac796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:23.006014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afca65-1778-4d03-963b-5b4470aac796 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:23.006014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afca65-1778-4d03-963b-5b4470aac796 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:23.006199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afca65-1778-4d03-963b-5b4470aac796 admin admin] PATH_INFO: `/v3/projects/001169f3507c4b379c27df1e65fc71d4/users/6f14d8d6af6b4444aa7b31b52b5411b1/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:23.026909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afca65-1778-4d03-963b-5b4470aac796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=001169f3507c4b379c27df1e65fc71d4, user_id=6f14d8d6af6b4444aa7b31b52b5411b1, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:23.030394 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afca65-1778-4d03-963b-5b4470aac796 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:23.106573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1201/2393] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:22 2022] PUT /identity/v3/projects/001169f3507c4b379c27df1e65fc71d4/users/6f14d8d6af6b4444aa7b31b52b5411b1/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:23.108447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a99e5-e3f5-479f-93ed-b3ef19784e32 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:23.125818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a99e5-e3f5-479f-93ed-b3ef19784e32 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:23.161873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:23.209391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6227ad-454d-4aa3-b696-ae7c039d401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:23.210010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6227ad-454d-4aa3-b696-ae7c039d401e admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:23.210310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6227ad-454d-4aa3-b696-ae7c039d401e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:23.210585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6227ad-454d-4aa3-b696-ae7c039d401e admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:23.211260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6227ad-454d-4aa3-b696-ae7c039d401e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:23.212894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6227ad-454d-4aa3-b696-ae7c039d401e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:23.224306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1193/2394] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:23 2022] GET /identity/v3/roles => generated 1427 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:23.228455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afca65-1778-4d03-963b-5b4470aac796 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:23.240133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afca65-1778-4d03-963b-5b4470aac796 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:23.244859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacef9f-7412-4557-a922-a733fd4bee59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:23.245513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacef9f-7412-4557-a922-a733fd4bee59 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:23.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacef9f-7412-4557-a922-a733fd4bee59 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:23.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bacef9f-7412-4557-a922-a733fd4bee59 admin admin] PATH_INFO: `/v3/projects/001169f3507c4b379c27df1e65fc71d4/users/6f14d8d6af6b4444aa7b31b52b5411b1/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:23.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacef9f-7412-4557-a922-a733fd4bee59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=001169f3507c4b379c27df1e65fc71d4, user_id=6f14d8d6af6b4444aa7b31b52b5411b1, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:23.253659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bacef9f-7412-4557-a922-a733fd4bee59 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:23.264633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1202/2395] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:23 2022] PUT /identity/v3/projects/001169f3507c4b379c27df1e65fc71d4/users/6f14d8d6af6b4444aa7b31b52b5411b1/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:23.275330 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8c6ff-fb79-4e38-ad0a-036aa69016ea None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:23.275621 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8c6ff-fb79-4e38-ad0a-036aa69016ea None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:23.275900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8c6ff-fb79-4e38-ad0a-036aa69016ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:23.309191 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7dd8c6ff-fb79-4e38-ad0a-036aa69016ea None None] MFA Rules not processed for user `6f14d8d6af6b4444aa7b31b52b5411b1`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:23.310458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7dd8c6ff-fb79-4e38-ad0a-036aa69016ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:23.382579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1194/2396] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:23 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:08:23.394745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacef9f-7412-4557-a922-a733fd4bee59 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:23.396474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6bacef9f-7412-4557-a922-a733fd4bee59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:23.459714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bacef9f-7412-4557-a922-a733fd4bee59 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:23.463514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b995e81-e324-4764-8ec9-310a8f8f3c00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:23.464122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b995e81-e324-4764-8ec9-310a8f8f3c00 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:23.464538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b995e81-e324-4764-8ec9-310a8f8f3c00 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:23.464820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b995e81-e324-4764-8ec9-310a8f8f3c00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:23.476168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b995e81-e324-4764-8ec9-310a8f8f3c00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:23.477528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b995e81-e324-4764-8ec9-310a8f8f3c00 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:23.487852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1203/2397] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:23 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:23.652103 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8c6ff-fb79-4e38-ad0a-036aa69016ea None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:23.663805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8c6ff-fb79-4e38-ad0a-036aa69016ea None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:23.667760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08b6492-1cea-4d1f-b874-8ad1775831b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:23.668415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b6492-1cea-4d1f-b874-8ad1775831b9 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:23.668688 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b6492-1cea-4d1f-b874-8ad1775831b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:23.668956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08b6492-1cea-4d1f-b874-8ad1775831b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:23.742723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b6492-1cea-4d1f-b874-8ad1775831b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:23.744395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08b6492-1cea-4d1f-b874-8ad1775831b9 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:23.754598 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1195/2398] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:24.320742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b995e81-e324-4764-8ec9-310a8f8f3c00 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:24.386675 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b995e81-e324-4764-8ec9-310a8f8f3c00 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:24.392141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:24.392775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:24.393052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:24.393322 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] PATH_INFO: `/v3/users/ee066ff2a0b4492993bb7ca8a900e571` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:24.394467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee066ff2a0b4492993bb7ca8a900e571)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:24.396459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:24.605448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee066ff2a0b4492993bb7ca8a900e571'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:24.606391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee066ff2a0b4492993bb7ca8a900e571'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:24.607274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee066ff2a0b4492993bb7ca8a900e571'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:24.699141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee066ff2a0b4492993bb7ca8a900e571'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:24.701613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:INSERT=6,keystone:SELECT=86,keystone:DELETE=8 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:24.722724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee066ff2a0b4492993bb7ca8a900e571'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:24.745156 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1204/2399] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:08:24 2022] DELETE /identity/v3/users/ee066ff2a0b4492993bb7ca8a900e571 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:24.753507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da328da-d60e-4157-b9b8-014c9cba4292 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:24.753507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da328da-d60e-4157-b9b8-014c9cba4292 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:24.753507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da328da-d60e-4157-b9b8-014c9cba4292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:24.781576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8da328da-d60e-4157-b9b8-014c9cba4292 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:24.783672 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8da328da-d60e-4157-b9b8-014c9cba4292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:24.840540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:24.866904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1196/2400] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:24.884872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:24.890574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:24.975100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ff6c5-4b9b-4a8f-9fd8-72734722d6b8 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:24.981144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd1218-5a2a-4e9a-afe3-a1261d0ecfe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:24.982147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd1218-5a2a-4e9a-afe3-a1261d0ecfe8 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:24.982601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd1218-5a2a-4e9a-afe3-a1261d0ecfe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:24.983103 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd1218-5a2a-4e9a-afe3-a1261d0ecfe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:24.999735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd1218-5a2a-4e9a-afe3-a1261d0ecfe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:25.002384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dd1218-5a2a-4e9a-afe3-a1261d0ecfe8 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:25.016946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1205/2401] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:25.862213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da328da-d60e-4157-b9b8-014c9cba4292 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:25.863911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8da328da-d60e-4157-b9b8-014c9cba4292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:25.880002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da328da-d60e-4157-b9b8-014c9cba4292 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:25.886954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:25.887629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:25.887911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:25.888204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] PATH_INFO: `/v3/projects/9fbd8722fc4b4ca6a60dfb6983906644` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:25.889311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fbd8722fc4b4ca6a60dfb6983906644)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:25.891811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:26.036864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fbd8722fc4b4ca6a60dfb6983906644', 'request_id': 'req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:26.043462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fbd8722fc4b4ca6a60dfb6983906644', 'request_id': 'req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:26.114544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fbd8722fc4b4ca6a60dfb6983906644', 'request_id': 'req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:26.134123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] The token cache is being invalidate because project 9fbd8722fc4b4ca6a60dfb6983906644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:08:26.134569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:26.135661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:26.137955 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1197/2402] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:08:25 2022] DELETE /identity/v3/projects/9fbd8722fc4b4ca6a60dfb6983906644 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:26.143099 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd1218-5a2a-4e9a-afe3-a1261d0ecfe8 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:26.154305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd1218-5a2a-4e9a-afe3-a1261d0ecfe8 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:26.227698 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:26.228326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:26.228603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:26.228870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] PATH_INFO: `/v3/users/6f14d8d6af6b4444aa7b31b52b5411b1` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:26.230052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f14d8d6af6b4444aa7b31b52b5411b1)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:26.231998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:26.528874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f14d8d6af6b4444aa7b31b52b5411b1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:26.529488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f14d8d6af6b4444aa7b31b52b5411b1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:26.530265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f14d8d6af6b4444aa7b31b52b5411b1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:26.636339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f14d8d6af6b4444aa7b31b52b5411b1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:26.653492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-029d7542-bf63-4356-be5e-973e7528ee88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f14d8d6af6b4444aa7b31b52b5411b1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:26.660689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:26.669596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1206/2403] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:08:26 2022] DELETE /identity/v3/users/6f14d8d6af6b4444aa7b31b52b5411b1 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:27.244285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:27.245934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:27.266421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d924ad-22fd-48f1-8a59-8688bd6f57cb admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:27.354110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:27.354734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:27.355008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:27.355306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] PATH_INFO: `/v3/projects/001169f3507c4b379c27df1e65fc71d4` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:27.356487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=001169f3507c4b379c27df1e65fc71d4)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:27.359136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:27.494618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '001169f3507c4b379c27df1e65fc71d4', 'request_id': 'req-7adb81b5-b3a7-4bcc-9a5a-69c356763428'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:27.502519 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '001169f3507c4b379c27df1e65fc71d4', 'request_id': 'req-7adb81b5-b3a7-4bcc-9a5a-69c356763428'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:27.568372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '001169f3507c4b379c27df1e65fc71d4', 'request_id': 'req-7adb81b5-b3a7-4bcc-9a5a-69c356763428'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:27.582085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] The token cache is being invalidate because project 001169f3507c4b379c27df1e65fc71d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:08:27.582362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:27.583065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7adb81b5-b3a7-4bcc-9a5a-69c356763428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:27.584567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1198/2404] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:08:27 2022] DELETE /identity/v3/projects/001169f3507c4b379c27df1e65fc71d4 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:27.590033 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5af0e-6ccd-481b-8bd1-2cc49fe33f66 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:27.590599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5af0e-6ccd-481b-8bd1-2cc49fe33f66 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:27.590599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5af0e-6ccd-481b-8bd1-2cc49fe33f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:27.612080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-e5f5af0e-6ccd-481b-8bd1-2cc49fe33f66 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:27.613295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e5f5af0e-6ccd-481b-8bd1-2cc49fe33f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:27.684856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1207/2405] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:27.693905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b880c2-a998-4eca-98e2-30746e3692e5 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:27.694185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b880c2-a998-4eca-98e2-30746e3692e5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:27.694459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b880c2-a998-4eca-98e2-30746e3692e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:27.703117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:27.716388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e3b880c2-a998-4eca-98e2-30746e3692e5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:27.717622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e3b880c2-a998-4eca-98e2-30746e3692e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:27.728041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1199/2406] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:27.733255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5af0e-6ccd-481b-8bd1-2cc49fe33f66 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:27.747901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5af0e-6ccd-481b-8bd1-2cc49fe33f66 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:27.753505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d106a-9091-48ba-816a-a0b2b4a0775b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:27.754126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d106a-9091-48ba-816a-a0b2b4a0775b admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:27.754445 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d106a-9091-48ba-816a-a0b2b4a0775b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:27.754720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d106a-9091-48ba-816a-a0b2b4a0775b admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:27.755406 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d106a-9091-48ba-816a-a0b2b4a0775b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:08:27.755708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d106a-9091-48ba-816a-a0b2b4a0775b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:27.757305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d106a-9091-48ba-816a-a0b2b4a0775b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:27.774266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1208/2407] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:08:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:08:27.778781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b880c2-a998-4eca-98e2-30746e3692e5 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:27.789919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b880c2-a998-4eca-98e2-30746e3692e5 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:27.795393 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3286798-abbb-453d-8c3c-2d2c33c59c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:27.797131 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286798-abbb-453d-8c3c-2d2c33c59c49 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:27.797515 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286798-abbb-453d-8c3c-2d2c33c59c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:27.797839 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286798-abbb-453d-8c3c-2d2c33c59c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:27.798752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3286798-abbb-453d-8c3c-2d2c33c59c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:27.801989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3286798-abbb-453d-8c3c-2d2c33c59c49 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:27.904814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1200/2408] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:08:27 2022] POST /identity/v3/projects => generated 378 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:27.909279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d106a-9091-48ba-816a-a0b2b4a0775b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:27.921678 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d106a-9091-48ba-816a-a0b2b4a0775b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:27.990348 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb376653-7ce2-4bff-9f59-dc8d8b2c309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:27.990965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb376653-7ce2-4bff-9f59-dc8d8b2c309a admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:27.991279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb376653-7ce2-4bff-9f59-dc8d8b2c309a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:27.991549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb376653-7ce2-4bff-9f59-dc8d8b2c309a admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:27.992172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb376653-7ce2-4bff-9f59-dc8d8b2c309a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:27.994106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb376653-7ce2-4bff-9f59-dc8d8b2c309a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:28.103984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1209/2409] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:08:27 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:28.105406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3286798-abbb-453d-8c3c-2d2c33c59c49 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:28.118179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3286798-abbb-453d-8c3c-2d2c33c59c49 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:28.126290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be134b-eb3d-4543-9936-5310bfe996c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:28.127520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be134b-eb3d-4543-9936-5310bfe996c1 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:28.127980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be134b-eb3d-4543-9936-5310bfe996c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:28.128291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be134b-eb3d-4543-9936-5310bfe996c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:28.128960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be134b-eb3d-4543-9936-5310bfe996c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:28.130687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85be134b-eb3d-4543-9936-5310bfe996c1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:28.147774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1201/2410] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:28 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:28.152155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb376653-7ce2-4bff-9f59-dc8d8b2c309a admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:28.168175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb376653-7ce2-4bff-9f59-dc8d8b2c309a admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:28.175222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69448a-e89a-4e4a-99b0-5f008f91e6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:28.175884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69448a-e89a-4e4a-99b0-5f008f91e6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:28.176163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69448a-e89a-4e4a-99b0-5f008f91e6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:28.176434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69448a-e89a-4e4a-99b0-5f008f91e6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:28.177085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69448a-e89a-4e4a-99b0-5f008f91e6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:28.179750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69448a-e89a-4e4a-99b0-5f008f91e6c7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:28.198924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1210/2411] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:28 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:28.204178 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be134b-eb3d-4543-9936-5310bfe996c1 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:28.216201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be134b-eb3d-4543-9936-5310bfe996c1 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:28.221803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b348d1-9cf9-47a5-b1c4-eb82e1f9ec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:28.222472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b348d1-9cf9-47a5-b1c4-eb82e1f9ec82 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:28.222753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b348d1-9cf9-47a5-b1c4-eb82e1f9ec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:28.223087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b348d1-9cf9-47a5-b1c4-eb82e1f9ec82 admin admin] PATH_INFO: `/v3/projects/7d2c92035cf74af4b5173bfb92ef2008/users/42b6d7288cad4158aebcff85c5f8dbde/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:28.251432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b348d1-9cf9-47a5-b1c4-eb82e1f9ec82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d2c92035cf74af4b5173bfb92ef2008, user_id=42b6d7288cad4158aebcff85c5f8dbde, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:28.255185 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b348d1-9cf9-47a5-b1c4-eb82e1f9ec82 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:28.338127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1202/2412] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:28 2022] PUT /identity/v3/projects/7d2c92035cf74af4b5173bfb92ef2008/users/42b6d7288cad4158aebcff85c5f8dbde/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:28.350660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641c1a8-ba83-4bca-b4fc-3a02f4cd34b3 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:28.350945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641c1a8-ba83-4bca-b4fc-3a02f4cd34b3 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:28.351248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641c1a8-ba83-4bca-b4fc-3a02f4cd34b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:28.388806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b641c1a8-ba83-4bca-b4fc-3a02f4cd34b3 None None] MFA Rules not processed for user `42b6d7288cad4158aebcff85c5f8dbde`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:28.390112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b641c1a8-ba83-4bca-b4fc-3a02f4cd34b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:28.448491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:28.472059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1211/2413] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:28.487755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b348d1-9cf9-47a5-b1c4-eb82e1f9ec82 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:28.489527 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-50b348d1-9cf9-47a5-b1c4-eb82e1f9ec82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:28.562980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b348d1-9cf9-47a5-b1c4-eb82e1f9ec82 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:28.567939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ff7b0-c153-40d5-b254-dd920182c67b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d62a4c07c4402ea5803f6e68fb1d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:28.568835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ff7b0-c153-40d5-b254-dd920182c67b service glance] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:28.569231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ff7b0-c153-40d5-b254-dd920182c67b service glance] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:28.569501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ff7b0-c153-40d5-b254-dd920182c67b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:28.579842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ff7b0-c153-40d5-b254-dd920182c67b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:28.581502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ff7b0-c153-40d5-b254-dd920182c67b service glance] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:28.591815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1203/2414] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:38.586993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:38.607815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:55.617299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641c1a8-ba83-4bca-b4fc-3a02f4cd34b3 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:55.682167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641c1a8-ba83-4bca-b4fc-3a02f4cd34b3 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:55.687802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:55.688428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:55.688700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:55.688965 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] PATH_INFO: `/v3/users/42b6d7288cad4158aebcff85c5f8dbde` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:55.690086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b6d7288cad4158aebcff85c5f8dbde)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:55.692031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:55.965231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b6d7288cad4158aebcff85c5f8dbde'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:55.965873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b6d7288cad4158aebcff85c5f8dbde'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:55.966480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b6d7288cad4158aebcff85c5f8dbde'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:56.036810 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b6d7288cad4158aebcff85c5f8dbde'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:56.050748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b6d7288cad4158aebcff85c5f8dbde'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:56.063344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1212/2415] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:08:55 2022] DELETE /identity/v3/users/42b6d7288cad4158aebcff85c5f8dbde => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:56.068326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb8270-e137-4847-8821-1be016298c3d None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:56.068822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb8270-e137-4847-8821-1be016298c3d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:56.068923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabb8270-e137-4847-8821-1be016298c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:56.088197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-dabb8270-e137-4847-8821-1be016298c3d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:56.089399 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-dabb8270-e137-4847-8821-1be016298c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:56.148968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1204/2416] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:56.169513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:56.171114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:56.231004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20079c1d-099f-420a-9850-a631219dad53 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:56.234524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1250357-2317-4daa-86d7-35a91dc0500a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:56.235155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1250357-2317-4daa-86d7-35a91dc0500a service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:56.235436 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1250357-2317-4daa-86d7-35a91dc0500a service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:56.235709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1250357-2317-4daa-86d7-35a91dc0500a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:56.244806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1250357-2317-4daa-86d7-35a91dc0500a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:56.246371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1250357-2317-4daa-86d7-35a91dc0500a service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:56.256888 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1213/2417] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:56 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:08:56.728216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb8270-e137-4847-8821-1be016298c3d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:56.735371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-dabb8270-e137-4847-8821-1be016298c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:56.745469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabb8270-e137-4847-8821-1be016298c3d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:56.750621 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:56.751767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:56.752073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:56.752493 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] PATH_INFO: `/v3/projects/7d2c92035cf74af4b5173bfb92ef2008` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:56.753851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d2c92035cf74af4b5173bfb92ef2008)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:56.755844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:56.910194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d2c92035cf74af4b5173bfb92ef2008', 'request_id': 'req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:56.916511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d2c92035cf74af4b5173bfb92ef2008', 'request_id': 'req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:57.005876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d2c92035cf74af4b5173bfb92ef2008', 'request_id': 'req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:57.017023 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] The token cache is being invalidate because project 7d2c92035cf74af4b5173bfb92ef2008 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:08:57.017309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:57.018167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-4f69fd62-36c9-4e2e-b381-ff1d6392f7d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:08:57.019763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1205/2418] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:08:56 2022] DELETE /identity/v3/projects/7d2c92035cf74af4b5173bfb92ef2008 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:57.024598 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74331d9-5fce-4776-86ce-ee53e3eb9dae None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:57.025205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74331d9-5fce-4776-86ce-ee53e3eb9dae None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:57.025319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74331d9-5fce-4776-86ce-ee53e3eb9dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:57.045605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d74331d9-5fce-4776-86ce-ee53e3eb9dae None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:57.046795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d74331d9-5fce-4776-86ce-ee53e3eb9dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:57.065246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:57.108104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1214/2419] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:08:57.116816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f738f1-c3c0-4442-82ee-357eba8db83f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:57.117282 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f738f1-c3c0-4442-82ee-357eba8db83f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:57.117409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f738f1-c3c0-4442-82ee-357eba8db83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:57.136942 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-28f738f1-c3c0-4442-82ee-357eba8db83f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:57.138168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-28f738f1-c3c0-4442-82ee-357eba8db83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:57.146818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1206/2420] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:57.151315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74331d9-5fce-4776-86ce-ee53e3eb9dae None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:57.162182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74331d9-5fce-4776-86ce-ee53e3eb9dae None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:57.166797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460e636-2e2e-404a-881d-a69e53077ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:57.167434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460e636-2e2e-404a-881d-a69e53077ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:57.167714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460e636-2e2e-404a-881d-a69e53077ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:57.167985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f460e636-2e2e-404a-881d-a69e53077ba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:57.168631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460e636-2e2e-404a-881d-a69e53077ba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:08:57.168930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460e636-2e2e-404a-881d-a69e53077ba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:57.170610 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f460e636-2e2e-404a-881d-a69e53077ba7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:57.183653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1215/2421] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:08:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:08:57.188165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f738f1-c3c0-4442-82ee-357eba8db83f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:57.197665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f738f1-c3c0-4442-82ee-357eba8db83f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:57.202477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee0ee0-ed2e-4c2a-a031-486b9ac31d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:57.203114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee0ee0-ed2e-4c2a-a031-486b9ac31d4e admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:57.203397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee0ee0-ed2e-4c2a-a031-486b9ac31d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:57.203696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee0ee0-ed2e-4c2a-a031-486b9ac31d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:57.204340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee0ee0-ed2e-4c2a-a031-486b9ac31d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:57.206916 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee0ee0-ed2e-4c2a-a031-486b9ac31d4e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:57.295987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1207/2422] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:08:57 2022] POST /identity/v3/projects => generated 384 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:57.300501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460e636-2e2e-404a-881d-a69e53077ba7 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:57.310632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f460e636-2e2e-404a-881d-a69e53077ba7 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:57.369305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c7726-9b79-4818-b32a-9ee573186f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:57.369911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c7726-9b79-4818-b32a-9ee573186f65 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:57.370192 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c7726-9b79-4818-b32a-9ee573186f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:57.370522 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c7726-9b79-4818-b32a-9ee573186f65 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:57.371179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c7726-9b79-4818-b32a-9ee573186f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:57.373164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c7726-9b79-4818-b32a-9ee573186f65 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:57.480776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1216/2423] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:08:57 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:08:57.484820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee0ee0-ed2e-4c2a-a031-486b9ac31d4e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:57.495409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee0ee0-ed2e-4c2a-a031-486b9ac31d4e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:57.500193 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf98da-efc3-4ae5-a8b8-f44e9aad5a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:57.500795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf98da-efc3-4ae5-a8b8-f44e9aad5a03 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:57.501070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf98da-efc3-4ae5-a8b8-f44e9aad5a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:57.501339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf98da-efc3-4ae5-a8b8-f44e9aad5a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:57.502468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf98da-efc3-4ae5-a8b8-f44e9aad5a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:57.504375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf98da-efc3-4ae5-a8b8-f44e9aad5a03 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:57.514961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1208/2424] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:57 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:57.519906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c7726-9b79-4818-b32a-9ee573186f65 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:57.530069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c7726-9b79-4818-b32a-9ee573186f65 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:57.535180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae44ced7-0bbb-4a8f-8321-a69538d3fdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:57.535800 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44ced7-0bbb-4a8f-8321-a69538d3fdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:57.536080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44ced7-0bbb-4a8f-8321-a69538d3fdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:57.536345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae44ced7-0bbb-4a8f-8321-a69538d3fdf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:57.537000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44ced7-0bbb-4a8f-8321-a69538d3fdf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:57.538569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae44ced7-0bbb-4a8f-8321-a69538d3fdf8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:57.550153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1217/2425] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:08:57 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:08:57.554616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf98da-efc3-4ae5-a8b8-f44e9aad5a03 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:57.565262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf98da-efc3-4ae5-a8b8-f44e9aad5a03 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:57.569940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d5c5-0ad1-4455-8031-1cf30886baf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:57.570590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d5c5-0ad1-4455-8031-1cf30886baf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:57.570868 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d5c5-0ad1-4455-8031-1cf30886baf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:57.571186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d5c5-0ad1-4455-8031-1cf30886baf6 admin admin] PATH_INFO: `/v3/projects/a1d12a8c096d4fea97010d12933a8ecd/users/09de3cb574604529b2d9dbe48d8f99b8/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:57.587065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708d5c5-0ad1-4455-8031-1cf30886baf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d12a8c096d4fea97010d12933a8ecd, user_id=09de3cb574604529b2d9dbe48d8f99b8, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:57.589945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708d5c5-0ad1-4455-8031-1cf30886baf6 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:57.660525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1209/2426] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:08:57 2022] PUT /identity/v3/projects/a1d12a8c096d4fea97010d12933a8ecd/users/09de3cb574604529b2d9dbe48d8f99b8/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:08:57.728157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a998411-def2-42ba-b752-8a534f3d75fe None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:57.728438 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a998411-def2-42ba-b752-8a534f3d75fe None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:57.728734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a998411-def2-42ba-b752-8a534f3d75fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:57.756793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-0a998411-def2-42ba-b752-8a534f3d75fe None None] MFA Rules not processed for user `09de3cb574604529b2d9dbe48d8f99b8`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:08:57.757931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0a998411-def2-42ba-b752-8a534f3d75fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:57.825276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1218/2427] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:08:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:08:57.836785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d5c5-0ad1-4455-8031-1cf30886baf6 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:08:57.838623 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7708d5c5-0ad1-4455-8031-1cf30886baf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:08:57.862691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:08:57.904173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d5c5-0ad1-4455-8031-1cf30886baf6 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:08:57.907945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1346898e-af97-487f-b8cf-bb6d6be04f4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:08:57.908658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346898e-af97-487f-b8cf-bb6d6be04f4d service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:08:57.909136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346898e-af97-487f-b8cf-bb6d6be04f4d service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:08:57.909416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1346898e-af97-487f-b8cf-bb6d6be04f4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:08:57.932216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346898e-af97-487f-b8cf-bb6d6be04f4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:08:57.932216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1346898e-af97-487f-b8cf-bb6d6be04f4d service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:08:57.933028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1210/2428] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:08:57 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:09:07.822030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:07.998141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=17 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:09.290171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a998411-def2-42ba-b752-8a534f3d75fe None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:09.359162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a998411-def2-42ba-b752-8a534f3d75fe None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:09.364012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:09.364645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:09.364924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:09.365197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] PATH_INFO: `/v3/users/09de3cb574604529b2d9dbe48d8f99b8` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:09.366371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09de3cb574604529b2d9dbe48d8f99b8)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:09.368363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:09.577821 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09de3cb574604529b2d9dbe48d8f99b8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:09:09.578559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09de3cb574604529b2d9dbe48d8f99b8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:09:09.579262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09de3cb574604529b2d9dbe48d8f99b8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:09:09.655363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09de3cb574604529b2d9dbe48d8f99b8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:09:09.668815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09de3cb574604529b2d9dbe48d8f99b8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:09:09.682921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1219/2429] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:09:09 2022] DELETE /identity/v3/users/09de3cb574604529b2d9dbe48d8f99b8 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:09:09.687700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957e6b6-a6a4-4561-bae2-2cf360761260 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:09.688201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957e6b6-a6a4-4561-bae2-2cf360761260 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:09.688398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957e6b6-a6a4-4561-bae2-2cf360761260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:09.707975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2957e6b6-a6a4-4561-bae2-2cf360761260 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:09.712732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2957e6b6-a6a4-4561-bae2-2cf360761260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:09.779211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1211/2430] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:09:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:09.789819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:09.791323 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:09.850767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd3de1-3fea-45b9-8f33-96565b837258 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:09.854407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d88f6-ad1b-40bd-b90b-d8f013160e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:09.855003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d88f6-ad1b-40bd-b90b-d8f013160e8e service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:09.855307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d88f6-ad1b-40bd-b90b-d8f013160e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:09.855574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d88f6-ad1b-40bd-b90b-d8f013160e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:09.866201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d88f6-ad1b-40bd-b90b-d8f013160e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:09.867668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d88f6-ad1b-40bd-b90b-d8f013160e8e service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:09.877930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1220/2431] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:09:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:09:10.319838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957e6b6-a6a4-4561-bae2-2cf360761260 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:10.321333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2957e6b6-a6a4-4561-bae2-2cf360761260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:10.334315 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957e6b6-a6a4-4561-bae2-2cf360761260 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:10.339510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:10.340150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:10.340438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:10.340728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] PATH_INFO: `/v3/projects/a1d12a8c096d4fea97010d12933a8ecd` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:10.341833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d12a8c096d4fea97010d12933a8ecd)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:10.344925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:10.457218 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d12a8c096d4fea97010d12933a8ecd', 'request_id': 'req-975b7fe4-bbc8-4807-8c23-905b472a8858'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:09:10.463475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d12a8c096d4fea97010d12933a8ecd', 'request_id': 'req-975b7fe4-bbc8-4807-8c23-905b472a8858'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:09:10.586039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d12a8c096d4fea97010d12933a8ecd', 'request_id': 'req-975b7fe4-bbc8-4807-8c23-905b472a8858'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:09:10.597565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] The token cache is being invalidate because project a1d12a8c096d4fea97010d12933a8ecd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:09:10.597851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:09:10.598573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-975b7fe4-bbc8-4807-8c23-905b472a8858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:09:10.600146 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1212/2432] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:09:10 2022] DELETE /identity/v3/projects/a1d12a8c096d4fea97010d12933a8ecd => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:09:10.605325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba265ccc-8394-4abd-b307-cfc766d1acf6 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:10.605808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba265ccc-8394-4abd-b307-cfc766d1acf6 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:10.607673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba265ccc-8394-4abd-b307-cfc766d1acf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:10.626874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ba265ccc-8394-4abd-b307-cfc766d1acf6 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:10.628125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ba265ccc-8394-4abd-b307-cfc766d1acf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:10.645536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:10.689984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1221/2433] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:09:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:10.697295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca66f8d-a1f6-4004-9f1c-c57e1a86f0e8 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:10.697800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca66f8d-a1f6-4004-9f1c-c57e1a86f0e8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:10.697901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca66f8d-a1f6-4004-9f1c-c57e1a86f0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:10.718759 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6ca66f8d-a1f6-4004-9f1c-c57e1a86f0e8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:10.720085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6ca66f8d-a1f6-4004-9f1c-c57e1a86f0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:10.728457 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1213/2434] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:09:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:10.739531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba265ccc-8394-4abd-b307-cfc766d1acf6 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:10.750055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba265ccc-8394-4abd-b307-cfc766d1acf6 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:10.754656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ca12a-93f3-4c33-93a1-74b84899c42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:10.755271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ca12a-93f3-4c33-93a1-74b84899c42a admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:10.755543 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ca12a-93f3-4c33-93a1-74b84899c42a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:10.755804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ca12a-93f3-4c33-93a1-74b84899c42a admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:10.756436 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ca12a-93f3-4c33-93a1-74b84899c42a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:09:10.756729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ca12a-93f3-4c33-93a1-74b84899c42a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:10.758300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ca12a-93f3-4c33-93a1-74b84899c42a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:10.771609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1222/2435] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:09:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:09:10.776197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca66f8d-a1f6-4004-9f1c-c57e1a86f0e8 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:10.786860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca66f8d-a1f6-4004-9f1c-c57e1a86f0e8 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:10.791677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add10b1-1c9d-44b4-871a-20295584f20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:10.792266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add10b1-1c9d-44b4-871a-20295584f20d admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:10.792537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add10b1-1c9d-44b4-871a-20295584f20d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:10.792800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add10b1-1c9d-44b4-871a-20295584f20d admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:10.793404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add10b1-1c9d-44b4-871a-20295584f20d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:10.795314 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add10b1-1c9d-44b4-871a-20295584f20d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:10.885182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1214/2436] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:09:10 2022] POST /identity/v3/projects => generated 386 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:09:10.889087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ca12a-93f3-4c33-93a1-74b84899c42a admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:10.900737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ca12a-93f3-4c33-93a1-74b84899c42a admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:10.961707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915076b-7f79-4662-9574-b016f2f30a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:10.962328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915076b-7f79-4662-9574-b016f2f30a61 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:10.962600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915076b-7f79-4662-9574-b016f2f30a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:10.962869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915076b-7f79-4662-9574-b016f2f30a61 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:10.963539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915076b-7f79-4662-9574-b016f2f30a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:10.965492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915076b-7f79-4662-9574-b016f2f30a61 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:11.075808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1223/2437] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:09:10 2022] POST /identity/v3/users => generated 294 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:09:11.081147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add10b1-1c9d-44b4-871a-20295584f20d admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:11.091836 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add10b1-1c9d-44b4-871a-20295584f20d admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:11.096523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7caed6-c468-4f41-920e-2a8a55d3471b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:11.097127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7caed6-c468-4f41-920e-2a8a55d3471b admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:11.097401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7caed6-c468-4f41-920e-2a8a55d3471b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:11.097669 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7caed6-c468-4f41-920e-2a8a55d3471b admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:11.098306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7caed6-c468-4f41-920e-2a8a55d3471b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:11.100048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7caed6-c468-4f41-920e-2a8a55d3471b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:11.110801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1215/2438] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:09:11 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:09:11.114914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915076b-7f79-4662-9574-b016f2f30a61 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:11.130309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915076b-7f79-4662-9574-b016f2f30a61 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:11.135159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b68e9-1fdb-4827-8231-f1b4f7df1e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:11.135828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b68e9-1fdb-4827-8231-f1b4f7df1e81 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:11.136070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b68e9-1fdb-4827-8231-f1b4f7df1e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:11.136341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b68e9-1fdb-4827-8231-f1b4f7df1e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:11.136973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b68e9-1fdb-4827-8231-f1b4f7df1e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:11.138647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b68e9-1fdb-4827-8231-f1b4f7df1e81 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:11.150458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1224/2439] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:09:11 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:09:11.154755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7caed6-c468-4f41-920e-2a8a55d3471b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:11.165058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7caed6-c468-4f41-920e-2a8a55d3471b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:11.169763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358f7a5-e658-47f7-8d26-0fd7c27e9bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:11.170408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358f7a5-e658-47f7-8d26-0fd7c27e9bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:11.170724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358f7a5-e658-47f7-8d26-0fd7c27e9bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:11.171230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358f7a5-e658-47f7-8d26-0fd7c27e9bbf admin admin] PATH_INFO: `/v3/projects/67112706d4df4183bc338732b237df6e/users/dfc1475f560d4151a0c569433544f36a/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:11.188308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b358f7a5-e658-47f7-8d26-0fd7c27e9bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=67112706d4df4183bc338732b237df6e, user_id=dfc1475f560d4151a0c569433544f36a, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:11.190992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b358f7a5-e658-47f7-8d26-0fd7c27e9bbf admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:11.284533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1216/2440] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:09:11 2022] PUT /identity/v3/projects/67112706d4df4183bc338732b237df6e/users/dfc1475f560d4151a0c569433544f36a/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:09:11.289690 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fce607-2663-46c9-8aa4-5c604eb7bf12 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:11.289976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fce607-2663-46c9-8aa4-5c604eb7bf12 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:11.290251 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fce607-2663-46c9-8aa4-5c604eb7bf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:11.310261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-79fce607-2663-46c9-8aa4-5c604eb7bf12 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:11.312001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-79fce607-2663-46c9-8aa4-5c604eb7bf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:11.370512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1225/2441] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:09:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:11.380996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358f7a5-e658-47f7-8d26-0fd7c27e9bbf admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:11.382686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b358f7a5-e658-47f7-8d26-0fd7c27e9bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:11.405154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:11.441105 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358f7a5-e658-47f7-8d26-0fd7c27e9bbf admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:11.444796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bf291-ba53-4ad5-af8c-3753e3c313f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:11.445401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf291-ba53-4ad5-af8c-3753e3c313f0 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:11.445677 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf291-ba53-4ad5-af8c-3753e3c313f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:11.445946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2bf291-ba53-4ad5-af8c-3753e3c313f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:11.455676 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2bf291-ba53-4ad5-af8c-3753e3c313f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:11.457091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2bf291-ba53-4ad5-af8c-3753e3c313f0 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:11.466347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1217/2442] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:09:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:09:14.406866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4ec38-b50b-458e-ba3c-3360e959c40a None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:14.406866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4ec38-b50b-458e-ba3c-3360e959c40a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:14.408995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb4ec38-b50b-458e-ba3c-3360e959c40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:14.432519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-dfb4ec38-b50b-458e-ba3c-3360e959c40a None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:14.433942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-dfb4ec38-b50b-458e-ba3c-3360e959c40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:14.466005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:14.529013 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1226/2443] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 05:09:14 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:14.691897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bf291-ba53-4ad5-af8c-3753e3c313f0 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:14.693924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-df2bf291-ba53-4ad5-af8c-3753e3c313f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:14.798109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2bf291-ba53-4ad5-af8c-3753e3c313f0 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:14.805547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eaf13-1769-421a-a313-564539c91fef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:14.806474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eaf13-1769-421a-a313-564539c91fef service swift] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:14.806799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eaf13-1769-421a-a313-564539c91fef service swift] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:14.807213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595eaf13-1769-421a-a313-564539c91fef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:14.818951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eaf13-1769-421a-a313-564539c91fef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:14.820906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595eaf13-1769-421a-a313-564539c91fef service swift] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:14.843259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1218/2444] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:09:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:09:18.107588 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff6e6-2151-4d62-8f0e-85f5b916a4e1 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:18.108017 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff6e6-2151-4d62-8f0e-85f5b916a4e1 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:18.108133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ff6e6-2151-4d62-8f0e-85f5b916a4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:18.154596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-7f7ff6e6-2151-4d62-8f0e-85f5b916a4e1 None None] MFA Rules not processed for user `dfc1475f560d4151a0c569433544f36a`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:18.155898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-7f7ff6e6-2151-4d62-8f0e-85f5b916a4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:18.284458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1227/2445] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:09:18 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:18.288774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eaf13-1769-421a-a313-564539c91fef service swift] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:18.305570 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595eaf13-1769-421a-a313-564539c91fef service swift] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:18.315220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ca64-5c54-4665-afaf-37acb73dd336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:18.315833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ca64-5c54-4665-afaf-37acb73dd336 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:18.316105 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ca64-5c54-4665-afaf-37acb73dd336 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:18.316554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ca64-5c54-4665-afaf-37acb73dd336 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:18.318082 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9ca64-5c54-4665-afaf-37acb73dd336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:18.320502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9ca64-5c54-4665-afaf-37acb73dd336 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:18.547236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1219/2446] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:09:18 2022] POST /identity/v3/projects => generated 386 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:09:18.553496 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ff6e6-2151-4d62-8f0e-85f5b916a4e1 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:18.573961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ff6e6-2151-4d62-8f0e-85f5b916a4e1 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:18.683487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347fbbf-83ec-4a0f-ad97-c7abf6c6deab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:18.684318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347fbbf-83ec-4a0f-ad97-c7abf6c6deab admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:18.684581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347fbbf-83ec-4a0f-ad97-c7abf6c6deab admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:18.684910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347fbbf-83ec-4a0f-ad97-c7abf6c6deab admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:18.685900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347fbbf-83ec-4a0f-ad97-c7abf6c6deab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:18.688971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347fbbf-83ec-4a0f-ad97-c7abf6c6deab admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:19.053182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1228/2447] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:09:18 2022] POST /identity/v3/users => generated 300 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:09:19.054652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ca64-5c54-4665-afaf-37acb73dd336 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:19.065383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ca64-5c54-4665-afaf-37acb73dd336 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:19.070458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0a374-c969-48c7-82b2-a57d991f1f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:19.071131 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a374-c969-48c7-82b2-a57d991f1f1a admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:19.071411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a374-c969-48c7-82b2-a57d991f1f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:19.071688 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0a374-c969-48c7-82b2-a57d991f1f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:19.072332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0a374-c969-48c7-82b2-a57d991f1f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:19.074062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0a374-c969-48c7-82b2-a57d991f1f1a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:19.087549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1220/2448] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:09:19 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:09:19.091087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347fbbf-83ec-4a0f-ad97-c7abf6c6deab admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:19.104469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347fbbf-83ec-4a0f-ad97-c7abf6c6deab admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:19.110684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6eaba-55d8-4a9f-8e9f-d8b81a1b87c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:19.111474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6eaba-55d8-4a9f-8e9f-d8b81a1b87c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:19.111789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6eaba-55d8-4a9f-8e9f-d8b81a1b87c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:19.112193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6eaba-55d8-4a9f-8e9f-d8b81a1b87c5 admin admin] PATH_INFO: `/v3/projects/08adf9792f4f45db9d1c98108116ec31/users/4ae6c4dd34e643999d652056fd911f97/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:19.139328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6eaba-55d8-4a9f-8e9f-d8b81a1b87c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08adf9792f4f45db9d1c98108116ec31, user_id=4ae6c4dd34e643999d652056fd911f97, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:19.144026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6eaba-55d8-4a9f-8e9f-d8b81a1b87c5 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:19.257892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1229/2449] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:09:19 2022] PUT /identity/v3/projects/08adf9792f4f45db9d1c98108116ec31/users/4ae6c4dd34e643999d652056fd911f97/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:09:19.262416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0a374-c969-48c7-82b2-a57d991f1f1a admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:19.274907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0a374-c969-48c7-82b2-a57d991f1f1a admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:19.359877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a6063-ffdb-4915-94ba-47140c023695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:19.360844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a6063-ffdb-4915-94ba-47140c023695 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:19.360844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a6063-ffdb-4915-94ba-47140c023695 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:19.361025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a6063-ffdb-4915-94ba-47140c023695 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:19.361656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a6063-ffdb-4915-94ba-47140c023695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:19.363431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a6063-ffdb-4915-94ba-47140c023695 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:19.385054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1221/2450] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:09:19 2022] GET /identity/v3/roles => generated 1427 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:09:19.391361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6eaba-55d8-4a9f-8e9f-d8b81a1b87c5 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:19.409016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6eaba-55d8-4a9f-8e9f-d8b81a1b87c5 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:19.415705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb747e77-68a5-4153-8576-b11179fc9421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:19.421648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb747e77-68a5-4153-8576-b11179fc9421 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:19.422655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb747e77-68a5-4153-8576-b11179fc9421 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:19.423700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb747e77-68a5-4153-8576-b11179fc9421 admin admin] PATH_INFO: `/v3/projects/08adf9792f4f45db9d1c98108116ec31/users/4ae6c4dd34e643999d652056fd911f97/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:19.428023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb747e77-68a5-4153-8576-b11179fc9421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08adf9792f4f45db9d1c98108116ec31, user_id=4ae6c4dd34e643999d652056fd911f97, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:19.432304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb747e77-68a5-4153-8576-b11179fc9421 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:19.459408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1230/2451] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:09:19 2022] PUT /identity/v3/projects/08adf9792f4f45db9d1c98108116ec31/users/4ae6c4dd34e643999d652056fd911f97/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:09:19.512487 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a6063-ffdb-4915-94ba-47140c023695 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:19.514294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-420a6063-ffdb-4915-94ba-47140c023695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:19.530797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:19.623336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a6063-ffdb-4915-94ba-47140c023695 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:19.767891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8394833-a005-41a0-9849-8112951c17ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:19.768529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8394833-a005-41a0-9849-8112951c17ba service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:19.769279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8394833-a005-41a0-9849-8112951c17ba service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:19.769423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8394833-a005-41a0-9849-8112951c17ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:19.771696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f8394833-a005-41a0-9849-8112951c17ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:19.915860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8394833-a005-41a0-9849-8112951c17ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:19.917310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8394833-a005-41a0-9849-8112951c17ba service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:19.953815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1222/2452] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:09:19 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:09:21.368292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb747e77-68a5-4153-8576-b11179fc9421 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:21.390314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb747e77-68a5-4153-8576-b11179fc9421 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:21.397921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f6865-3ed7-4993-bd53-a3fa73b36a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:21.399404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f6865-3ed7-4993-bd53-a3fa73b36a45 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:21.400024 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f6865-3ed7-4993-bd53-a3fa73b36a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:21.400527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4f6865-3ed7-4993-bd53-a3fa73b36a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:21.402458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1f4f6865-3ed7-4993-bd53-a3fa73b36a45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:21.425389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f6865-3ed7-4993-bd53-a3fa73b36a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:21.427057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4f6865-3ed7-4993-bd53-a3fa73b36a45 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:21.435739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:21.442670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1231/2453] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:09:21 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:09:27.072591 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24002c-154a-4d33-a7f5-882446d48e8a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:27.073164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24002c-154a-4d33-a7f5-882446d48e8a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:27.073164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24002c-154a-4d33-a7f5-882446d48e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:27.150023 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ef24002c-154a-4d33-a7f5-882446d48e8a None None] MFA Rules not processed for user `4ae6c4dd34e643999d652056fd911f97`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:27.151319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ef24002c-154a-4d33-a7f5-882446d48e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:27.212654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:27.262382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1223/2454] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:09:27 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:27.270960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f6865-3ed7-4993-bd53-a3fa73b36a45 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:27.272191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1f4f6865-3ed7-4993-bd53-a3fa73b36a45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:27.361528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4f6865-3ed7-4993-bd53-a3fa73b36a45 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:27.370631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91f506-cab9-43e4-82e6-3b5a376cb2c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d62a4c07c4402ea5803f6e68fb1d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:27.388301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91f506-cab9-43e4-82e6-3b5a376cb2c9 service glance] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:27.389089 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91f506-cab9-43e4-82e6-3b5a376cb2c9 service glance] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:27.389550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91f506-cab9-43e4-82e6-3b5a376cb2c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:27.480934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91f506-cab9-43e4-82e6-3b5a376cb2c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:27.482815 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91f506-cab9-43e4-82e6-3b5a376cb2c9 service glance] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:27.499771 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1232/2455] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:09:27 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:09:27.872951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0eab1-aa70-4d6f-8149-5c36571d68a4 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:27.873412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0eab1-aa70-4d6f-8149-5c36571d68a4 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:27.873789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0eab1-aa70-4d6f-8149-5c36571d68a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:27.895668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-17f0eab1-aa70-4d6f-8149-5c36571d68a4 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:27.917962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-17f0eab1-aa70-4d6f-8149-5c36571d68a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:27.962398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1224/2456] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 05:09:27 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:27.975787 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91f506-cab9-43e4-82e6-3b5a376cb2c9 service glance] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:28.070797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91f506-cab9-43e4-82e6-3b5a376cb2c9 service glance] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:28.079149 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e05028-1986-4382-a317-5accd40f0735 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:28.080094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e05028-1986-4382-a317-5accd40f0735 service swift] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:28.080537 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e05028-1986-4382-a317-5accd40f0735 service swift] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:28.080959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e05028-1986-4382-a317-5accd40f0735 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:28.095287 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e05028-1986-4382-a317-5accd40f0735 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:28.097972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e05028-1986-4382-a317-5accd40f0735 service swift] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:28.112365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1233/2457] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:09:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:09:35.778557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0eab1-aa70-4d6f-8149-5c36571d68a4 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:35.780311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-17f0eab1-aa70-4d6f-8149-5c36571d68a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:35.857826 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0eab1-aa70-4d6f-8149-5c36571d68a4 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:35.862308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8474bb0-57a6-4156-94e1-f502b9671e37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:35.863238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8474bb0-57a6-4156-94e1-f502b9671e37 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:35.863523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8474bb0-57a6-4156-94e1-f502b9671e37 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:35.863799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8474bb0-57a6-4156-94e1-f502b9671e37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:35.865787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b8474bb0-57a6-4156-94e1-f502b9671e37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:35.984785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8474bb0-57a6-4156-94e1-f502b9671e37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:35.987214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8474bb0-57a6-4156-94e1-f502b9671e37 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:36.021831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:36.026725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1225/2458] 10.222.0.105 () {60 vars in 1401 bytes} [Wed Aug 24 05:09:35 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3274 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:09:37.725832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab173a-2d31-4da9-9792-186fc6ef9d6e None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:37.726426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab173a-2d31-4da9-9792-186fc6ef9d6e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:37.726687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab173a-2d31-4da9-9792-186fc6ef9d6e None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:37.729016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1234/2459] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 05:09:37 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 24 05:09:37.738522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415846d-3f7f-4d0e-a15e-d7ac87f79c47 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:37.739158 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415846d-3f7f-4d0e-a15e-d7ac87f79c47 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:37.739444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415846d-3f7f-4d0e-a15e-d7ac87f79c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:37.794160 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7415846d-3f7f-4d0e-a15e-d7ac87f79c47 None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:37.796352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7415846d-3f7f-4d0e-a15e-d7ac87f79c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:37.809068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1226/2460] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 05:09:37 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:37.821563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab173a-2d31-4da9-9792-186fc6ef9d6e None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:37.824056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b7ab173a-2d31-4da9-9792-186fc6ef9d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:37.851355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab173a-2d31-4da9-9792-186fc6ef9d6e None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:37.861166 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b6c7c-260b-4c8b-bb72-fd552bf97507 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d62a4c07c4402ea5803f6e68fb1d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:37.861933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b6c7c-260b-4c8b-bb72-fd552bf97507 service glance] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:37.862270 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b6c7c-260b-4c8b-bb72-fd552bf97507 service glance] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:37.863188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b6c7c-260b-4c8b-bb72-fd552bf97507 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:37.890292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b6c7c-260b-4c8b-bb72-fd552bf97507 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:37.896059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b6c7c-260b-4c8b-bb72-fd552bf97507 service glance] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:37.912602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1235/2461] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:09:37 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:09:38.119797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bc42f-f08d-470e-a92b-40ba95651d9f None None] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:38.120331 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bc42f-f08d-470e-a92b-40ba95651d9f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:38.120756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bc42f-f08d-470e-a92b-40ba95651d9f None None] PATH_INFO: `/` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:38.122540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1227/2462] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 05:09:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 05:09:38.129796 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55260a7d-5660-4b73-b0c7-022f851c5aa4 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:38.132652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55260a7d-5660-4b73-b0c7-022f851c5aa4 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:38.132652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55260a7d-5660-4b73-b0c7-022f851c5aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:38.179933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-55260a7d-5660-4b73-b0c7-022f851c5aa4 None None] MFA Rules not processed for user `44d2b4015ac74b8b8d0a674419f34cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:38.181158 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-55260a7d-5660-4b73-b0c7-022f851c5aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:38.188778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1236/2463] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 05:09:38 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:38.201320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bc42f-f08d-470e-a92b-40ba95651d9f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:38.217648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bc42f-f08d-470e-a92b-40ba95651d9f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:38.221321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6888bb-93b8-411b-82d9-2b060a6c00e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:38.221925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6888bb-93b8-411b-82d9-2b060a6c00e6 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:38.222210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6888bb-93b8-411b-82d9-2b060a6c00e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:38.222482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6888bb-93b8-411b-82d9-2b060a6c00e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:38.230188 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6888bb-93b8-411b-82d9-2b060a6c00e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:38.231656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6888bb-93b8-411b-82d9-2b060a6c00e6 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:38.241846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1228/2464] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:09:38 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:09:38.352218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33093f-0ab0-45f2-8815-5ca29c11b5b1 None None] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:38.352549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33093f-0ab0-45f2-8815-5ca29c11b5b1 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:38.352895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33093f-0ab0-45f2-8815-5ca29c11b5b1 None None] PATH_INFO: `/` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:38.354471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1237/2465] 10.222.0.105 () {54 vars in 926 bytes} [Wed Aug 24 05:09:38 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 24 05:09:38.360568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c418119-fea3-4302-8ba1-b4d795b8c0db None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:38.361084 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c418119-fea3-4302-8ba1-b4d795b8c0db None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:38.361421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c418119-fea3-4302-8ba1-b4d795b8c0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:38.383264 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-0c418119-fea3-4302-8ba1-b4d795b8c0db None None] MFA Rules not processed for user `3c822c8f23b74bae84769046fdb85db4`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:38.384468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0c418119-fea3-4302-8ba1-b4d795b8c0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:38.417610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1229/2466] 10.222.0.105 () {58 vars in 1023 bytes} [Wed Aug 24 05:09:38 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:38.429763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33093f-0ab0-45f2-8815-5ca29c11b5b1 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:38.441582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33093f-0ab0-45f2-8815-5ca29c11b5b1 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:38.445129 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef580816-11ab-46a6-ab64-003a017ae6bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:38.445724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef580816-11ab-46a6-ab64-003a017ae6bc service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:38.446001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef580816-11ab-46a6-ab64-003a017ae6bc service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:38.446271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef580816-11ab-46a6-ab64-003a017ae6bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:38.453339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:38.457138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef580816-11ab-46a6-ab64-003a017ae6bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:38.458545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef580816-11ab-46a6-ab64-003a017ae6bc service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:38.469596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1238/2467] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:09:38 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:09:39.889130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de856a71-5e6b-49f9-b3e5-84bc1dffb576 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:39.889680 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de856a71-5e6b-49f9-b3e5-84bc1dffb576 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:39.889680 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de856a71-5e6b-49f9-b3e5-84bc1dffb576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:39.911513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-de856a71-5e6b-49f9-b3e5-84bc1dffb576 None None] MFA Rules not processed for user `dfc1475f560d4151a0c569433544f36a`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:39.912939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-de856a71-5e6b-49f9-b3e5-84bc1dffb576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:39.922858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1230/2468] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:09:39 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:41.047848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8888b-3595-4705-a162-e1deadafa637 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:41.047848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8888b-3595-4705-a162-e1deadafa637 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:41.047848 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a8888b-3595-4705-a162-e1deadafa637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:41.068243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c2a8888b-3595-4705-a162-e1deadafa637 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:41.069479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c2a8888b-3595-4705-a162-e1deadafa637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:41.078364 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1239/2469] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 05:09:41 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:41.090847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de856a71-5e6b-49f9-b3e5-84bc1dffb576 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:41.104209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de856a71-5e6b-49f9-b3e5-84bc1dffb576 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:41.109329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2719c-e995-47d8-806d-0b8f92eb33b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:41.110705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2719c-e995-47d8-806d-0b8f92eb33b1 service swift] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:41.110985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2719c-e995-47d8-806d-0b8f92eb33b1 service swift] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:41.111472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2719c-e995-47d8-806d-0b8f92eb33b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:41.120253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2719c-e995-47d8-806d-0b8f92eb33b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:41.121855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2719c-e995-47d8-806d-0b8f92eb33b1 service swift] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:41.130637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1231/2470] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:09:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:09:51.063718 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=6 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:51.126959 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=27 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:09:58.641222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb8dcc-048f-4623-a6b6-894f8be4cae8 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:58.642093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb8dcc-048f-4623-a6b6-894f8be4cae8 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:58.642576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb8dcc-048f-4623-a6b6-894f8be4cae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:58.722185 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-cfdb8dcc-048f-4623-a6b6-894f8be4cae8 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:09:58.723665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cfdb8dcc-048f-4623-a6b6-894f8be4cae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:09:58.750804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1240/2471] 10.222.0.105 () {58 vars in 1014 bytes} [Wed Aug 24 05:09:58 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:09:58.776350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2719c-e995-47d8-806d-0b8f92eb33b1 service swift] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:09:58.802123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2719c-e995-47d8-806d-0b8f92eb33b1 service swift] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:09:58.807683 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f4260-d8cd-44f4-83f5-f39b0ab23de1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:09:58.808217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f4260-d8cd-44f4-83f5-f39b0ab23de1 service swift] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:09:58.808493 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f4260-d8cd-44f4-83f5-f39b0ab23de1 service swift] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:09:58.808763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f4260-d8cd-44f4-83f5-f39b0ab23de1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:09:58.819140 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f4260-d8cd-44f4-83f5-f39b0ab23de1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:09:58.821052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f4260-d8cd-44f4-83f5-f39b0ab23de1 service swift] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:09:58.830864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1232/2472] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:09:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:10:07.598697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb8dcc-048f-4623-a6b6-894f8be4cae8 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:07.598697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb8dcc-048f-4623-a6b6-894f8be4cae8 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:07.598697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a115291-b80b-487b-b411-64815a0d7ecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:07.598697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a115291-b80b-487b-b411-64815a0d7ecc service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:07.599540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a115291-b80b-487b-b411-64815a0d7ecc service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:07.599540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a115291-b80b-487b-b411-64815a0d7ecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:07.599540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9a115291-b80b-487b-b411-64815a0d7ecc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:07.614973 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a115291-b80b-487b-b411-64815a0d7ecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:07.616495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a115291-b80b-487b-b411-64815a0d7ecc service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:07.625614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1241/2473] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:07 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:10:08.436463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f4260-d8cd-44f4-83f5-f39b0ab23de1 service swift] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:08.437895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-113f4260-d8cd-44f4-83f5-f39b0ab23de1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:08.455634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f4260-d8cd-44f4-83f5-f39b0ab23de1 service swift] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:08.461900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45e66c-2504-429d-9bfc-b9ef8a198193 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:08.462525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45e66c-2504-429d-9bfc-b9ef8a198193 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:08.462801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45e66c-2504-429d-9bfc-b9ef8a198193 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:08.463095 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45e66c-2504-429d-9bfc-b9ef8a198193 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:08.464877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9f45e66c-2504-429d-9bfc-b9ef8a198193 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:08.480866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45e66c-2504-429d-9bfc-b9ef8a198193 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:08.482526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45e66c-2504-429d-9bfc-b9ef8a198193 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:08.495066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1233/2474] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:08 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:10:15.036138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce77bc3-a35e-442b-83c2-2c180f00c3fa None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:15.036633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce77bc3-a35e-442b-83c2-2c180f00c3fa None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:15.036970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce77bc3-a35e-442b-83c2-2c180f00c3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:15.059263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-9ce77bc3-a35e-442b-83c2-2c180f00c3fa None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:10:15.060470 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9ce77bc3-a35e-442b-83c2-2c180f00c3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:15.067814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1242/2475] 10.222.0.105 () {58 vars in 972 bytes} [Wed Aug 24 05:10:15 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:10:15.090076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45e66c-2504-429d-9bfc-b9ef8a198193 service nova] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:15.103401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45e66c-2504-429d-9bfc-b9ef8a198193 service nova] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:15.108627 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82afa11-6222-4a05-939a-e5f68f685b05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:15.109239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82afa11-6222-4a05-939a-e5f68f685b05 service swift] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:15.109521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82afa11-6222-4a05-939a-e5f68f685b05 service swift] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:15.109949 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82afa11-6222-4a05-939a-e5f68f685b05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:15.119736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82afa11-6222-4a05-939a-e5f68f685b05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:15.121339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82afa11-6222-4a05-939a-e5f68f685b05 service swift] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:15.130380 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1234/2476] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:10:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:10:17.965472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce77bc3-a35e-442b-83c2-2c180f00c3fa None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:17.980724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce77bc3-a35e-442b-83c2-2c180f00c3fa None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:17.988402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5942234-0492-4569-9b49-504636250dec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:17.988402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5942234-0492-4569-9b49-504636250dec service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:17.988402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5942234-0492-4569-9b49-504636250dec service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:17.988402 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5942234-0492-4569-9b49-504636250dec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:17.989200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b5942234-0492-4569-9b49-504636250dec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:18.001805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5942234-0492-4569-9b49-504636250dec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:18.003422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5942234-0492-4569-9b49-504636250dec service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:18.014763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1243/2477] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:17 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:10:25.126783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=22 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:26.854273 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82afa11-6222-4a05-939a-e5f68f685b05 service swift] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:26.864463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82afa11-6222-4a05-939a-e5f68f685b05 service swift] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:26.869228 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef75331-38da-428c-b802-dd1d7e77685f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:26.869824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef75331-38da-428c-b802-dd1d7e77685f service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:26.870095 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef75331-38da-428c-b802-dd1d7e77685f service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:26.870359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef75331-38da-428c-b802-dd1d7e77685f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:26.872230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-aef75331-38da-428c-b802-dd1d7e77685f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:26.945326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef75331-38da-428c-b802-dd1d7e77685f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:26.946776 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef75331-38da-428c-b802-dd1d7e77685f service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:26.956748 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1235/2478] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:10:28.010392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=33 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:34.557090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5942234-0492-4569-9b49-504636250dec service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:34.566662 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5942234-0492-4569-9b49-504636250dec service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:34.570708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61be23-4b51-4728-a002-792e5130a667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:34.571332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61be23-4b51-4728-a002-792e5130a667 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:34.571601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61be23-4b51-4728-a002-792e5130a667 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:34.571909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61be23-4b51-4728-a002-792e5130a667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:34.573775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ea61be23-4b51-4728-a002-792e5130a667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:34.640216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61be23-4b51-4728-a002-792e5130a667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:34.641656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61be23-4b51-4728-a002-792e5130a667 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:34.652092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1244/2479] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:10:36.952912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=28 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:41.309119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef75331-38da-428c-b802-dd1d7e77685f service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:41.319189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef75331-38da-428c-b802-dd1d7e77685f service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:41.323319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69780efe-c2bb-40e2-8e7b-83c164387e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:41.323934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69780efe-c2bb-40e2-8e7b-83c164387e1c service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:41.324213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69780efe-c2bb-40e2-8e7b-83c164387e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:41.324483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69780efe-c2bb-40e2-8e7b-83c164387e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:41.334858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69780efe-c2bb-40e2-8e7b-83c164387e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:41.336368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69780efe-c2bb-40e2-8e7b-83c164387e1c service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:41.346526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1236/2480] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:41 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:10:43.909091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61be23-4b51-4728-a002-792e5130a667 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:43.920186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61be23-4b51-4728-a002-792e5130a667 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:43.924105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5d447-c568-4b81-b638-95bf3d850eb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:43.924730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5d447-c568-4b81-b638-95bf3d850eb2 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:43.925004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5d447-c568-4b81-b638-95bf3d850eb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:43.925279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b5d447-c568-4b81-b638-95bf3d850eb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:43.927452 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-09b5d447-c568-4b81-b638-95bf3d850eb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:44.056489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5d447-c568-4b81-b638-95bf3d850eb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:44.057948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b5d447-c568-4b81-b638-95bf3d850eb2 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:44.087036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1245/2481] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:10:44.535090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69780efe-c2bb-40e2-8e7b-83c164387e1c service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:44.539172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-69780efe-c2bb-40e2-8e7b-83c164387e1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:44.557950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69780efe-c2bb-40e2-8e7b-83c164387e1c service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:44.563745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fd0b2-1fd1-4e1a-adb7-bd3019cd897f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:44.565585 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fd0b2-1fd1-4e1a-adb7-bd3019cd897f service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:44.570880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fd0b2-1fd1-4e1a-adb7-bd3019cd897f service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:44.570880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fd0b2-1fd1-4e1a-adb7-bd3019cd897f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:44.570880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c99fd0b2-1fd1-4e1a-adb7-bd3019cd897f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:44.603727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fd0b2-1fd1-4e1a-adb7-bd3019cd897f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:44.616850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fd0b2-1fd1-4e1a-adb7-bd3019cd897f service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:44.633018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1237/2482] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:44 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:10:46.598685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5d447-c568-4b81-b638-95bf3d850eb2 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:46.601305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-09b5d447-c568-4b81-b638-95bf3d850eb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:46.621648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b5d447-c568-4b81-b638-95bf3d850eb2 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:46.631945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:46.631945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:46.631945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:46.631945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] PATH_INFO: `/v3/users/bfb6ab7a2f344a069010a3385d2fcf59` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:46.632417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfb6ab7a2f344a069010a3385d2fcf59)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:46.633159 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:46.908705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfb6ab7a2f344a069010a3385d2fcf59'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:46.909414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfb6ab7a2f344a069010a3385d2fcf59'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:46.910011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfb6ab7a2f344a069010a3385d2fcf59'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:47.032526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfb6ab7a2f344a069010a3385d2fcf59'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:47.038168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:47.047355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfb6ab7a2f344a069010a3385d2fcf59'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:47.060301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1246/2483] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:10:46 2022] DELETE /identity/v3/users/bfb6ab7a2f344a069010a3385d2fcf59 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:10:47.340795 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fd0b2-1fd1-4e1a-adb7-bd3019cd897f service nova] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:47.355062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fd0b2-1fd1-4e1a-adb7-bd3019cd897f service nova] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:47.376152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:47.379582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:47.446415 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:47.449985 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:47.450269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:47.450548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] PATH_INFO: `/v3/projects/f6a6379a0607459187075dfcd968d706` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:47.452356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6a6379a0607459187075dfcd968d706)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:47.454287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:47.462962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f740c1-aedc-4929-b55a-2ae9fe85b49f admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:47.468366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528f6b-c301-4622-960b-d030ba2ec250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:47.469096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528f6b-c301-4622-960b-d030ba2ec250 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:47.469383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528f6b-c301-4622-960b-d030ba2ec250 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:47.469653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528f6b-c301-4622-960b-d030ba2ec250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:47.472789 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-46528f6b-c301-4622-960b-d030ba2ec250 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:47.488373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528f6b-c301-4622-960b-d030ba2ec250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:47.489867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528f6b-c301-4622-960b-d030ba2ec250 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:47.501400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1247/2484] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:10:47.597796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6a6379a0607459187075dfcd968d706', 'request_id': 'req-6d1e655f-3400-446a-a7e5-6ad2614cad52'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:47.605328 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6a6379a0607459187075dfcd968d706', 'request_id': 'req-6d1e655f-3400-446a-a7e5-6ad2614cad52'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:47.726570 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6a6379a0607459187075dfcd968d706', 'request_id': 'req-6d1e655f-3400-446a-a7e5-6ad2614cad52'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:47.745382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] The token cache is being invalidate because project f6a6379a0607459187075dfcd968d706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:10:47.745684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:47.746352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:47.748714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1238/2485] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:10:47 2022] DELETE /identity/v3/projects/f6a6379a0607459187075dfcd968d706 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:10:47.752723 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528f6b-c301-4622-960b-d030ba2ec250 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:47.755038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-46528f6b-c301-4622-960b-d030ba2ec250 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:47.770411 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528f6b-c301-4622-960b-d030ba2ec250 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:47.844955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:47.845874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:47.846337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:47.846897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] PATH_INFO: `/v3/users/7ca2dda5987649ae91db22bf20852355` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:47.848442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca2dda5987649ae91db22bf20852355)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:47.850943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:48.109366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca2dda5987649ae91db22bf20852355'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:48.110185 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca2dda5987649ae91db22bf20852355'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:48.111231 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca2dda5987649ae91db22bf20852355'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:48.207274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca2dda5987649ae91db22bf20852355'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:48.223048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca2dda5987649ae91db22bf20852355'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:48.238540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1248/2486] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:10:47 2022] DELETE /identity/v3/users/7ca2dda5987649ae91db22bf20852355 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:10:48.727285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:48.727285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:48.778069 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e655f-3400-446a-a7e5-6ad2614cad52 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:48.857549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:48.858478 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:48.858744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:48.859219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] PATH_INFO: `/v3/projects/cae251b8db4b41ed8a7c98aa77a40a49` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:48.860703 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae251b8db4b41ed8a7c98aa77a40a49)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:48.862798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:48.872222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:49.036763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:49.072038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae251b8db4b41ed8a7c98aa77a40a49', 'request_id': 'req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:49.079945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae251b8db4b41ed8a7c98aa77a40a49', 'request_id': 'req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:49.168678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae251b8db4b41ed8a7c98aa77a40a49', 'request_id': 'req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:49.181615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] The token cache is being invalidate because project cae251b8db4b41ed8a7c98aa77a40a49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:10:49.181904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:49.182646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:49.188268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1239/2487] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:10:48 2022] DELETE /identity/v3/projects/cae251b8db4b41ed8a7c98aa77a40a49 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:10:52.354726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:52.356277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:52.371381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d64ba4-413b-4b03-9059-022c2f78fd99 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:52.435916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:52.436765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:52.437184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:52.437590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] PATH_INFO: `/v3/users/b11da4dac5804252b7310be096ae5d53` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:52.439342 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b11da4dac5804252b7310be096ae5d53)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:52.442485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:52.673883 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b11da4dac5804252b7310be096ae5d53'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:52.674569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b11da4dac5804252b7310be096ae5d53'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:52.675201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b11da4dac5804252b7310be096ae5d53'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:52.758637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b11da4dac5804252b7310be096ae5d53'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:52.772580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b11da4dac5804252b7310be096ae5d53'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:52.785649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1249/2488] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:10:52 2022] DELETE /identity/v3/users/b11da4dac5804252b7310be096ae5d53 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:10:53.062639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:53.064204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:53.078147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02eb8c2-de2e-49ba-bc8e-dbd995d16728 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:53.168961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:53.179925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:53.179925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:53.179925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] PATH_INFO: `/v3/projects/bd6824ae6edd43fcbddb58581134098c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:53.179925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd6824ae6edd43fcbddb58581134098c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:53.179925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:53.242741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:53.244097 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:53.314974 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0c500-31fc-4c59-a68c-529e31fefacd admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:53.318884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a14b5-c0e4-40b5-9334-c464aa627e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:53.319547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a14b5-c0e4-40b5-9334-c464aa627e59 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:53.319823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a14b5-c0e4-40b5-9334-c464aa627e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:53.320093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a14b5-c0e4-40b5-9334-c464aa627e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:53.321930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fb4a14b5-c0e4-40b5-9334-c464aa627e59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:53.334714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a14b5-c0e4-40b5-9334-c464aa627e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:53.336350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a14b5-c0e4-40b5-9334-c464aa627e59 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:53.348043 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1250/2489] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:10:53.374561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd6824ae6edd43fcbddb58581134098c', 'request_id': 'req-a503bb4b-4110-4c14-81fe-d0a0d34441d2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:53.381642 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd6824ae6edd43fcbddb58581134098c', 'request_id': 'req-a503bb4b-4110-4c14-81fe-d0a0d34441d2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:53.515950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd6824ae6edd43fcbddb58581134098c', 'request_id': 'req-a503bb4b-4110-4c14-81fe-d0a0d34441d2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:53.531847 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] The token cache is being invalidate because project bd6824ae6edd43fcbddb58581134098c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:10:53.532167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:53.532843 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a503bb4b-4110-4c14-81fe-d0a0d34441d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:53.535410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1240/2490] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:10:53 2022] DELETE /identity/v3/projects/bd6824ae6edd43fcbddb58581134098c => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:10:53.541293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7e42f-d378-4c38-977e-b63f27152c79 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:53.541836 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7e42f-d378-4c38-977e-b63f27152c79 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:53.541836 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7e42f-d378-4c38-977e-b63f27152c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:53.563200 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-50d7e42f-d378-4c38-977e-b63f27152c79 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:10:53.564284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-50d7e42f-d378-4c38-977e-b63f27152c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:53.580126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:53.640386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1251/2491] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:10:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:10:53.650009 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15fe6d-229d-40d4-ab1a-422a040c6f09 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:53.650461 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15fe6d-229d-40d4-ab1a-422a040c6f09 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:53.655756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c15fe6d-229d-40d4-ab1a-422a040c6f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:53.672393 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7c15fe6d-229d-40d4-ab1a-422a040c6f09 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:10:53.673705 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7c15fe6d-229d-40d4-ab1a-422a040c6f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:53.685055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1241/2492] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:10:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:10:53.689519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7e42f-d378-4c38-977e-b63f27152c79 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:53.700910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d7e42f-d378-4c38-977e-b63f27152c79 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:53.707437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e78c9-886e-420a-b6c7-1462e16225a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:53.708073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e78c9-886e-420a-b6c7-1462e16225a2 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:53.708371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e78c9-886e-420a-b6c7-1462e16225a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:53.708732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e78c9-886e-420a-b6c7-1462e16225a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:53.709557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e78c9-886e-420a-b6c7-1462e16225a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:10:53.709998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e78c9-886e-420a-b6c7-1462e16225a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:53.711680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e78c9-886e-420a-b6c7-1462e16225a2 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:53.726282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1252/2493] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:10:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:10:53.730500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15fe6d-229d-40d4-ab1a-422a040c6f09 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:53.742339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c15fe6d-229d-40d4-ab1a-422a040c6f09 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:53.747414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d0ae8-8230-48ba-abf4-9e42f1a51e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:53.748049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d0ae8-8230-48ba-abf4-9e42f1a51e1f admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:53.748327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d0ae8-8230-48ba-abf4-9e42f1a51e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:53.748595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d0ae8-8230-48ba-abf4-9e42f1a51e1f admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:53.749246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d0ae8-8230-48ba-abf4-9e42f1a51e1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:53.763220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d0ae8-8230-48ba-abf4-9e42f1a51e1f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:53.783393 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:53.895624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1242/2494] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:10:53 2022] POST /identity/v3/projects => generated 376 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:10:53.899904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e78c9-886e-420a-b6c7-1462e16225a2 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:53.913229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e78c9-886e-420a-b6c7-1462e16225a2 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:53.986912 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5abae0-c14d-4a42-a5cc-de287a8fd058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:53.987971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5abae0-c14d-4a42-a5cc-de287a8fd058 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:53.988540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5abae0-c14d-4a42-a5cc-de287a8fd058 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:53.989114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5abae0-c14d-4a42-a5cc-de287a8fd058 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:53.990216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5abae0-c14d-4a42-a5cc-de287a8fd058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:53.993400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5abae0-c14d-4a42-a5cc-de287a8fd058 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:54.092638 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1253/2495] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:10:53 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:10:54.094469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d0ae8-8230-48ba-abf4-9e42f1a51e1f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:54.106559 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d0ae8-8230-48ba-abf4-9e42f1a51e1f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:54.117425 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9fb23-a33b-43ed-a940-2903d4c7dc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:54.118062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9fb23-a33b-43ed-a940-2903d4c7dc16 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:54.118391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9fb23-a33b-43ed-a940-2903d4c7dc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:54.118666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9fb23-a33b-43ed-a940-2903d4c7dc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:54.119339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9fb23-a33b-43ed-a940-2903d4c7dc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:54.122834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9fb23-a33b-43ed-a940-2903d4c7dc16 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:54.138870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1243/2496] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:10:54 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:10:54.143812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5abae0-c14d-4a42-a5cc-de287a8fd058 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:54.155379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5abae0-c14d-4a42-a5cc-de287a8fd058 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:54.160252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cac5c6-a4d2-4c1e-96ea-e73dbcc9a009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:54.160862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cac5c6-a4d2-4c1e-96ea-e73dbcc9a009 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:54.161160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cac5c6-a4d2-4c1e-96ea-e73dbcc9a009 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:54.161434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cac5c6-a4d2-4c1e-96ea-e73dbcc9a009 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:54.162078 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cac5c6-a4d2-4c1e-96ea-e73dbcc9a009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:54.164285 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cac5c6-a4d2-4c1e-96ea-e73dbcc9a009 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:54.180467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1254/2497] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:10:54 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:10:54.181726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9fb23-a33b-43ed-a940-2903d4c7dc16 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:54.192565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9fb23-a33b-43ed-a940-2903d4c7dc16 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:54.198046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f57328-ca4f-46de-accd-3739232aacc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:54.198722 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f57328-ca4f-46de-accd-3739232aacc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:54.199000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f57328-ca4f-46de-accd-3739232aacc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:54.205910 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f57328-ca4f-46de-accd-3739232aacc2 admin admin] PATH_INFO: `/v3/projects/9e0bf8c9c5d44bed8b7b2bcd19d6c6c1/users/a8677a70f5ed4b96bf32018b9638fac2/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:54.216815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f57328-ca4f-46de-accd-3739232aacc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e0bf8c9c5d44bed8b7b2bcd19d6c6c1, user_id=a8677a70f5ed4b96bf32018b9638fac2, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:54.219517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f57328-ca4f-46de-accd-3739232aacc2 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:54.333909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1244/2498] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:10:54 2022] PUT /identity/v3/projects/9e0bf8c9c5d44bed8b7b2bcd19d6c6c1/users/a8677a70f5ed4b96bf32018b9638fac2/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:10:54.345885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3763607-9195-42b2-94a6-156da2dc1e98 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:54.346338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3763607-9195-42b2-94a6-156da2dc1e98 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:54.346875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3763607-9195-42b2-94a6-156da2dc1e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:54.387184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b3763607-9195-42b2-94a6-156da2dc1e98 None None] MFA Rules not processed for user `a8677a70f5ed4b96bf32018b9638fac2`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:10:54.389308 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b3763607-9195-42b2-94a6-156da2dc1e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:54.475214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1255/2499] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:10:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:10:54.480907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f57328-ca4f-46de-accd-3739232aacc2 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:54.562073 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f57328-ca4f-46de-accd-3739232aacc2 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:54.569824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0d7dd-83ec-456a-bb5d-bb54c799ea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:54.585003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0d7dd-83ec-456a-bb5d-bb54c799ea5f admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:54.585003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0d7dd-83ec-456a-bb5d-bb54c799ea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:54.585003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0d7dd-83ec-456a-bb5d-bb54c799ea5f admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:54.585003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0d7dd-83ec-456a-bb5d-bb54c799ea5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:54.585003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0d7dd-83ec-456a-bb5d-bb54c799ea5f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:54.708770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1245/2500] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:10:54 2022] POST /identity/v3/projects => generated 374 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:10:54.715768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3763607-9195-42b2-94a6-156da2dc1e98 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:54.731591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3763607-9195-42b2-94a6-156da2dc1e98 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:54.746387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:54.796910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4c37f-6dfb-4cd5-a052-60b8e4a73ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:54.797439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4c37f-6dfb-4cd5-a052-60b8e4a73ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:54.797748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4c37f-6dfb-4cd5-a052-60b8e4a73ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:54.798016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4c37f-6dfb-4cd5-a052-60b8e4a73ed9 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:54.798688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4c37f-6dfb-4cd5-a052-60b8e4a73ed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:54.801058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4c37f-6dfb-4cd5-a052-60b8e4a73ed9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:54.893319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1256/2501] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:10:54 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:10:54.897978 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0d7dd-83ec-456a-bb5d-bb54c799ea5f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:54.908932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0d7dd-83ec-456a-bb5d-bb54c799ea5f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:54.914339 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10f4e4-db5a-4898-bee6-620602ec9b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:54.914954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10f4e4-db5a-4898-bee6-620602ec9b7c admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:54.915259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10f4e4-db5a-4898-bee6-620602ec9b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:54.915525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10f4e4-db5a-4898-bee6-620602ec9b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:54.916161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10f4e4-db5a-4898-bee6-620602ec9b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:54.917775 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10f4e4-db5a-4898-bee6-620602ec9b7c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:54.934842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1246/2502] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:10:54 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:10:54.936041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4c37f-6dfb-4cd5-a052-60b8e4a73ed9 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:54.947275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4c37f-6dfb-4cd5-a052-60b8e4a73ed9 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:54.954368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fefae5-c068-404a-b239-b8c74443c277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:54.955463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fefae5-c068-404a-b239-b8c74443c277 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:54.955958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fefae5-c068-404a-b239-b8c74443c277 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:54.956460 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fefae5-c068-404a-b239-b8c74443c277 admin admin] PATH_INFO: `/v3/projects/19f97cb9ab4b4593ae04ca27b1759cf4/users/5bfe1c81cf294d9a956f54c238fb6578/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:54.979852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fefae5-c068-404a-b239-b8c74443c277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f97cb9ab4b4593ae04ca27b1759cf4, user_id=5bfe1c81cf294d9a956f54c238fb6578, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:54.982539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fefae5-c068-404a-b239-b8c74443c277 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:55.082519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1257/2503] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:10:54 2022] PUT /identity/v3/projects/19f97cb9ab4b4593ae04ca27b1759cf4/users/5bfe1c81cf294d9a956f54c238fb6578/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:10:55.083656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10f4e4-db5a-4898-bee6-620602ec9b7c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:55.094354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10f4e4-db5a-4898-bee6-620602ec9b7c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:55.167524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ae33b-b96f-4b0e-b445-f4008455a3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:55.168175 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ae33b-b96f-4b0e-b445-f4008455a3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:55.168593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ae33b-b96f-4b0e-b445-f4008455a3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:55.168972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ae33b-b96f-4b0e-b445-f4008455a3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:55.169736 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ae33b-b96f-4b0e-b445-f4008455a3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:55.172247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ae33b-b96f-4b0e-b445-f4008455a3a0 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:55.187157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1247/2504] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:10:55 2022] GET /identity/v3/roles => generated 1427 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:10:55.196088 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fefae5-c068-404a-b239-b8c74443c277 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:55.208423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fefae5-c068-404a-b239-b8c74443c277 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:55.214074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5d8fa-8605-4ca8-ab49-c20cf7053043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:55.220181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d8fa-8605-4ca8-ab49-c20cf7053043 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:55.220181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d8fa-8605-4ca8-ab49-c20cf7053043 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:55.220181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c5d8fa-8605-4ca8-ab49-c20cf7053043 admin admin] PATH_INFO: `/v3/projects/19f97cb9ab4b4593ae04ca27b1759cf4/users/5bfe1c81cf294d9a956f54c238fb6578/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:55.220181 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5d8fa-8605-4ca8-ab49-c20cf7053043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19f97cb9ab4b4593ae04ca27b1759cf4, user_id=5bfe1c81cf294d9a956f54c238fb6578, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:55.220617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c5d8fa-8605-4ca8-ab49-c20cf7053043 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:55.232791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1258/2505] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:10:55 2022] PUT /identity/v3/projects/19f97cb9ab4b4593ae04ca27b1759cf4/users/5bfe1c81cf294d9a956f54c238fb6578/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:10:55.244313 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970a52a-5b50-4956-ab06-b4a6cc32c474 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:55.249053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970a52a-5b50-4956-ab06-b4a6cc32c474 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:55.249053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970a52a-5b50-4956-ab06-b4a6cc32c474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:55.281960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e970a52a-5b50-4956-ab06-b4a6cc32c474 None None] MFA Rules not processed for user `5bfe1c81cf294d9a956f54c238fb6578`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:10:55.283395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e970a52a-5b50-4956-ab06-b4a6cc32c474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:55.306019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:55.370939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1248/2506] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:10:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:10:55.382425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5d8fa-8605-4ca8-ab49-c20cf7053043 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:55.384300 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-63c5d8fa-8605-4ca8-ab49-c20cf7053043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:55.467474 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c5d8fa-8605-4ca8-ab49-c20cf7053043 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:55.471900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dffae-266e-4754-9c55-c066389fd3ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:55.472621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dffae-266e-4754-9c55-c066389fd3ec service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:55.472899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dffae-266e-4754-9c55-c066389fd3ec service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:55.473175 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329dffae-266e-4754-9c55-c066389fd3ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:55.544777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dffae-266e-4754-9c55-c066389fd3ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:55.546173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329dffae-266e-4754-9c55-c066389fd3ec service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:55.557641 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1259/2507] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:10:57.363442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970a52a-5b50-4956-ab06-b4a6cc32c474 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:57.364948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e970a52a-5b50-4956-ab06-b4a6cc32c474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:57.447818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e970a52a-5b50-4956-ab06-b4a6cc32c474 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:57.453486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:57.454409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:57.454806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:57.455279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] PATH_INFO: `/v3/users/e45111c49aa74f76874d3a0c747034ba` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:57.472200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e45111c49aa74f76874d3a0c747034ba)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:57.474043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:57.692142 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e45111c49aa74f76874d3a0c747034ba'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:57.692798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e45111c49aa74f76874d3a0c747034ba'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:57.693407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e45111c49aa74f76874d3a0c747034ba'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:57.758693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e45111c49aa74f76874d3a0c747034ba'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:57.772956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f630cce5-2210-48d6-a6a9-021527fecf59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e45111c49aa74f76874d3a0c747034ba'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:57.786997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1249/2508] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:10:57 2022] DELETE /identity/v3/users/e45111c49aa74f76874d3a0c747034ba => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:10:58.102907 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dffae-266e-4754-9c55-c066389fd3ec service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:58.116134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329dffae-266e-4754-9c55-c066389fd3ec service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:58.138831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:58.177411 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:58.178602 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:58.178960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:58.179265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] PATH_INFO: `/v3/projects/fad6d7352bc144929743ca7373a41800` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:58.198504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fad6d7352bc144929743ca7373a41800)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:58.200633 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:58.373028 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fad6d7352bc144929743ca7373a41800', 'request_id': 'req-c0bb82ab-4999-449d-b05b-b2065ede2fb0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:58.379725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fad6d7352bc144929743ca7373a41800', 'request_id': 'req-c0bb82ab-4999-449d-b05b-b2065ede2fb0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:58.386455 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fad6d7352bc144929743ca7373a41800', 'request_id': 'req-c0bb82ab-4999-449d-b05b-b2065ede2fb0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:58.489731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] The token cache is being invalidate because project fad6d7352bc144929743ca7373a41800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:10:58.490026 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:58.491366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c0bb82ab-4999-449d-b05b-b2065ede2fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:10:58.492880 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1260/2509] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:10:58 2022] DELETE /identity/v3/projects/fad6d7352bc144929743ca7373a41800 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:10:58.498162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec870c7a-11f2-4622-ba7c-03af2392f815 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:58.498674 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec870c7a-11f2-4622-ba7c-03af2392f815 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:58.498784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec870c7a-11f2-4622-ba7c-03af2392f815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:58.518426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ec870c7a-11f2-4622-ba7c-03af2392f815 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:10:58.519671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ec870c7a-11f2-4622-ba7c-03af2392f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:58.539944 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:58.581552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1250/2510] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:10:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:10:58.589865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192ebb-1ac8-44c4-80fe-4f94438e506a None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:58.590149 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192ebb-1ac8-44c4-80fe-4f94438e506a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:58.590418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8192ebb-1ac8-44c4-80fe-4f94438e506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:58.613856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-d8192ebb-1ac8-44c4-80fe-4f94438e506a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:10:58.615091 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d8192ebb-1ac8-44c4-80fe-4f94438e506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:58.626239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1261/2511] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:10:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:10:58.630173 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec870c7a-11f2-4622-ba7c-03af2392f815 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:58.640975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec870c7a-11f2-4622-ba7c-03af2392f815 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:58.646183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ed2fa-7493-421a-988c-708d23da1479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:58.646837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ed2fa-7493-421a-988c-708d23da1479 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:58.647187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ed2fa-7493-421a-988c-708d23da1479 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:58.651607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ed2fa-7493-421a-988c-708d23da1479 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:58.651607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ed2fa-7493-421a-988c-708d23da1479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:10:58.651607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ed2fa-7493-421a-988c-708d23da1479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:58.651607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ed2fa-7493-421a-988c-708d23da1479 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:58.664026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1251/2512] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:10:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:10:58.668623 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8192ebb-1ac8-44c4-80fe-4f94438e506a None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:58.680152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8192ebb-1ac8-44c4-80fe-4f94438e506a None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:58.685080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b43a-efa5-4aed-83cd-24fb71a58a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:58.685701 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b43a-efa5-4aed-83cd-24fb71a58a51 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:58.685982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b43a-efa5-4aed-83cd-24fb71a58a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:58.686247 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7b43a-efa5-4aed-83cd-24fb71a58a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:58.686886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7b43a-efa5-4aed-83cd-24fb71a58a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:58.688853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7b43a-efa5-4aed-83cd-24fb71a58a51 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:58.831339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1262/2513] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:10:58 2022] POST /identity/v3/projects => generated 372 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:10:58.836773 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ed2fa-7493-421a-988c-708d23da1479 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:58.847631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ed2fa-7493-421a-988c-708d23da1479 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:58.906789 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4dd4e-c222-40d3-9e87-acbeb8e9266b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:58.907368 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4dd4e-c222-40d3-9e87-acbeb8e9266b admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:58.907646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4dd4e-c222-40d3-9e87-acbeb8e9266b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:58.907917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4dd4e-c222-40d3-9e87-acbeb8e9266b admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:58.908579 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4dd4e-c222-40d3-9e87-acbeb8e9266b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:58.910563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd4dd4e-c222-40d3-9e87-acbeb8e9266b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:59.030381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1252/2514] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:10:58 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:10:59.034720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b43a-efa5-4aed-83cd-24fb71a58a51 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:59.046139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7b43a-efa5-4aed-83cd-24fb71a58a51 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:59.050653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036a65e-5c44-4be4-abcc-f52ce52fce72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:59.051524 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036a65e-5c44-4be4-abcc-f52ce52fce72 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:59.051795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036a65e-5c44-4be4-abcc-f52ce52fce72 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:59.052066 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036a65e-5c44-4be4-abcc-f52ce52fce72 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:59.052725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036a65e-5c44-4be4-abcc-f52ce52fce72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:59.054591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036a65e-5c44-4be4-abcc-f52ce52fce72 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:59.066421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1263/2515] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:10:59 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:10:59.071219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4dd4e-c222-40d3-9e87-acbeb8e9266b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:59.081507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4dd4e-c222-40d3-9e87-acbeb8e9266b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:59.087267 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06532f11-f8a0-4240-96aa-b6292df3ad16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:59.087887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532f11-f8a0-4240-96aa-b6292df3ad16 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:59.088163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532f11-f8a0-4240-96aa-b6292df3ad16 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:59.088435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532f11-f8a0-4240-96aa-b6292df3ad16 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:59.089076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06532f11-f8a0-4240-96aa-b6292df3ad16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:59.090753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06532f11-f8a0-4240-96aa-b6292df3ad16 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:59.102287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1253/2516] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:10:59 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:10:59.106593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036a65e-5c44-4be4-abcc-f52ce52fce72 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:59.117098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036a65e-5c44-4be4-abcc-f52ce52fce72 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:59.121661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72762f28-8871-4e18-8ddd-e1d63efb5f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:59.122294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72762f28-8871-4e18-8ddd-e1d63efb5f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:59.122568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72762f28-8871-4e18-8ddd-e1d63efb5f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:59.122833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72762f28-8871-4e18-8ddd-e1d63efb5f9a admin admin] PATH_INFO: `/v3/projects/abd030a80a194419a3766a2924cf1139/users/8ce18713b2ed4b7ead32aec832eff5ba/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:59.138187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72762f28-8871-4e18-8ddd-e1d63efb5f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd030a80a194419a3766a2924cf1139, user_id=8ce18713b2ed4b7ead32aec832eff5ba, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:59.140949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72762f28-8871-4e18-8ddd-e1d63efb5f9a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:59.227593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1264/2517] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:10:59 2022] PUT /identity/v3/projects/abd030a80a194419a3766a2924cf1139/users/8ce18713b2ed4b7ead32aec832eff5ba/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:10:59.239286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc647-f44e-43ed-81cb-03ffa03afe3d None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:59.239286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc647-f44e-43ed-81cb-03ffa03afe3d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:59.239286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc647-f44e-43ed-81cb-03ffa03afe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:59.267117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-a78cc647-f44e-43ed-81cb-03ffa03afe3d None None] MFA Rules not processed for user `8ce18713b2ed4b7ead32aec832eff5ba`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:10:59.268342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a78cc647-f44e-43ed-81cb-03ffa03afe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:59.334086 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1254/2518] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:10:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:10:59.345839 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72762f28-8871-4e18-8ddd-e1d63efb5f9a admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:10:59.347610 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-72762f28-8871-4e18-8ddd-e1d63efb5f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:10:59.392603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:10:59.424052 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72762f28-8871-4e18-8ddd-e1d63efb5f9a admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:10:59.430934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf1892-89b2-41d2-9263-36be1f722138 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:10:59.431567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf1892-89b2-41d2-9263-36be1f722138 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:10:59.431844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf1892-89b2-41d2-9263-36be1f722138 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:10:59.432115 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf1892-89b2-41d2-9263-36be1f722138 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:10:59.444442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf1892-89b2-41d2-9263-36be1f722138 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:10:59.445962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cf1892-89b2-41d2-9263-36be1f722138 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:10:59.455242 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1265/2519] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:10:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:03.110462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cc647-f44e-43ed-81cb-03ffa03afe3d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:03.111931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a78cc647-f44e-43ed-81cb-03ffa03afe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:03.138780 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:03.172262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cc647-f44e-43ed-81cb-03ffa03afe3d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:03.175997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869e01d1-abd4-4ae7-ae7e-fc30cadba3c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:03.177006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e01d1-abd4-4ae7-ae7e-fc30cadba3c8 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:03.177288 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e01d1-abd4-4ae7-ae7e-fc30cadba3c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:03.177555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869e01d1-abd4-4ae7-ae7e-fc30cadba3c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:03.179321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-869e01d1-abd4-4ae7-ae7e-fc30cadba3c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:03.243473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e01d1-abd4-4ae7-ae7e-fc30cadba3c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:03.245155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869e01d1-abd4-4ae7-ae7e-fc30cadba3c8 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:03.255673 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1255/2520] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:04.923350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf1892-89b2-41d2-9263-36be1f722138 service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:05.004154 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf1892-89b2-41d2-9263-36be1f722138 service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:05.008846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:05.009503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:05.009780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:05.010144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] PATH_INFO: `/v3/users/8ce18713b2ed4b7ead32aec832eff5ba` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:05.011746 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ce18713b2ed4b7ead32aec832eff5ba)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:05.013677 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:05.213449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ce18713b2ed4b7ead32aec832eff5ba'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:05.214123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ce18713b2ed4b7ead32aec832eff5ba'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:05.214730 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ce18713b2ed4b7ead32aec832eff5ba'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:05.298832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ce18713b2ed4b7ead32aec832eff5ba'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:05.313513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ce18713b2ed4b7ead32aec832eff5ba'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:05.326578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1266/2521] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:04 2022] DELETE /identity/v3/users/8ce18713b2ed4b7ead32aec832eff5ba => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:05.331329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c8c0b-ebd4-4a01-b5b3-df42889c104d None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:05.332083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c8c0b-ebd4-4a01-b5b3-df42889c104d None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:05.332083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c8c0b-ebd4-4a01-b5b3-df42889c104d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:05.352232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7e5c8c0b-ebd4-4a01-b5b3-df42889c104d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:05.354026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7e5c8c0b-ebd4-4a01-b5b3-df42889c104d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:05.422269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1256/2522] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:05.434530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:05.436439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:05.495725 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:05.507309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588dee52-c72a-43a0-8c65-be04c41f9b93 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:05.511229 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd91e55-91b7-4f6f-b080-ac394bf9991c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:05.512060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd91e55-91b7-4f6f-b080-ac394bf9991c service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:05.512345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd91e55-91b7-4f6f-b080-ac394bf9991c service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:05.512619 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd91e55-91b7-4f6f-b080-ac394bf9991c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:05.532092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd91e55-91b7-4f6f-b080-ac394bf9991c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:05.534011 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd91e55-91b7-4f6f-b080-ac394bf9991c service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:05.545975 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1267/2523] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:05.971512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c8c0b-ebd4-4a01-b5b3-df42889c104d None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:05.971635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7e5c8c0b-ebd4-4a01-b5b3-df42889c104d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:05.986885 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c8c0b-ebd4-4a01-b5b3-df42889c104d None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:05.993032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:05.993760 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:05.994141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:05.994443 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] PATH_INFO: `/v3/projects/abd030a80a194419a3766a2924cf1139` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:05.995757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd030a80a194419a3766a2924cf1139)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:05.997675 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:06.139853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd030a80a194419a3766a2924cf1139', 'request_id': 'req-b6791f87-7c9b-4626-953b-59420f677b23'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:06.146154 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd030a80a194419a3766a2924cf1139', 'request_id': 'req-b6791f87-7c9b-4626-953b-59420f677b23'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:06.213525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd030a80a194419a3766a2924cf1139', 'request_id': 'req-b6791f87-7c9b-4626-953b-59420f677b23'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:06.218065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:06.225107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] The token cache is being invalidate because project abd030a80a194419a3766a2924cf1139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:06.225398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:06.226043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-b6791f87-7c9b-4626-953b-59420f677b23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:06.227565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1257/2524] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:05 2022] DELETE /identity/v3/projects/abd030a80a194419a3766a2924cf1139 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:06.233187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55812d3-4cc4-4c63-b3c1-44448a14332b None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:06.233668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55812d3-4cc4-4c63-b3c1-44448a14332b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:06.233793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55812d3-4cc4-4c63-b3c1-44448a14332b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:06.253557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c55812d3-4cc4-4c63-b3c1-44448a14332b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:06.254763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c55812d3-4cc4-4c63-b3c1-44448a14332b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:06.315157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1268/2525] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:06.324731 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8aa6e-8adc-4dfe-8eb7-ca0865f95433 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:06.325011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8aa6e-8adc-4dfe-8eb7-ca0865f95433 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:06.325285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d8aa6e-8adc-4dfe-8eb7-ca0865f95433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:06.345563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-a7d8aa6e-8adc-4dfe-8eb7-ca0865f95433 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:06.346679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a7d8aa6e-8adc-4dfe-8eb7-ca0865f95433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:06.355243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1258/2526] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:06.359566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55812d3-4cc4-4c63-b3c1-44448a14332b None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:06.369325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55812d3-4cc4-4c63-b3c1-44448a14332b None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:06.373937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fff99-ebb8-4c2e-89c0-8525ca31d9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:06.374538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fff99-ebb8-4c2e-89c0-8525ca31d9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:06.374814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fff99-ebb8-4c2e-89c0-8525ca31d9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:06.375117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fff99-ebb8-4c2e-89c0-8525ca31d9d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:06.375757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fff99-ebb8-4c2e-89c0-8525ca31d9d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:06.376054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fff99-ebb8-4c2e-89c0-8525ca31d9d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:06.377655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fff99-ebb8-4c2e-89c0-8525ca31d9d6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:06.390104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1269/2527] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:11:06.394496 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8aa6e-8adc-4dfe-8eb7-ca0865f95433 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:06.405040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d8aa6e-8adc-4dfe-8eb7-ca0865f95433 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:06.410112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65625-a2a3-4f2a-8f00-8d4216cb4a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:06.410715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65625-a2a3-4f2a-8f00-8d4216cb4a6c admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:06.410989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65625-a2a3-4f2a-8f00-8d4216cb4a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:06.411289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65625-a2a3-4f2a-8f00-8d4216cb4a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:06.411915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65625-a2a3-4f2a-8f00-8d4216cb4a6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:06.413820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65625-a2a3-4f2a-8f00-8d4216cb4a6c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:06.542904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1259/2528] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:06 2022] POST /identity/v3/projects => generated 378 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:06.547164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fff99-ebb8-4c2e-89c0-8525ca31d9d6 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:06.558037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fff99-ebb8-4c2e-89c0-8525ca31d9d6 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:06.615739 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7254cf-6389-461c-9faa-bb5661859dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:06.616262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7254cf-6389-461c-9faa-bb5661859dad admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:06.616537 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7254cf-6389-461c-9faa-bb5661859dad admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:06.616805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7254cf-6389-461c-9faa-bb5661859dad admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:06.617427 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7254cf-6389-461c-9faa-bb5661859dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:06.619905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7254cf-6389-461c-9faa-bb5661859dad admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:06.695933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1270/2529] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:06 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:06.700671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65625-a2a3-4f2a-8f00-8d4216cb4a6c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:06.711187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65625-a2a3-4f2a-8f00-8d4216cb4a6c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:06.716633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bca33-f83a-4267-88d6-c381e1176332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:06.717244 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bca33-f83a-4267-88d6-c381e1176332 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:06.717521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bca33-f83a-4267-88d6-c381e1176332 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:06.717793 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bca33-f83a-4267-88d6-c381e1176332 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:06.718436 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bca33-f83a-4267-88d6-c381e1176332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:06.720063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bca33-f83a-4267-88d6-c381e1176332 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:06.730108 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1260/2530] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:06 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:06.734235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7254cf-6389-461c-9faa-bb5661859dad admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:06.744817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7254cf-6389-461c-9faa-bb5661859dad admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:06.749491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502befb-21b3-40d2-be89-9cd1ef1faee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:06.750081 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502befb-21b3-40d2-be89-9cd1ef1faee6 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:06.750355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502befb-21b3-40d2-be89-9cd1ef1faee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:06.750621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502befb-21b3-40d2-be89-9cd1ef1faee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:06.751320 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502befb-21b3-40d2-be89-9cd1ef1faee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:06.752893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502befb-21b3-40d2-be89-9cd1ef1faee6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:06.763743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1271/2531] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:06 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:06.767977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bca33-f83a-4267-88d6-c381e1176332 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:06.778958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bca33-f83a-4267-88d6-c381e1176332 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:06.784016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820b793-b71e-4979-98be-b3a86692f237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:06.784660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b793-b71e-4979-98be-b3a86692f237 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:06.784934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b793-b71e-4979-98be-b3a86692f237 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:06.785202 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820b793-b71e-4979-98be-b3a86692f237 admin admin] PATH_INFO: `/v3/projects/d4e559327d314c59b6345dae98803866/users/46e0ce6e66864773a04e81a796e637cb/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:06.800030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820b793-b71e-4979-98be-b3a86692f237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4e559327d314c59b6345dae98803866, user_id=46e0ce6e66864773a04e81a796e637cb, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:06.802909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820b793-b71e-4979-98be-b3a86692f237 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:06.861553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1261/2532] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:06 2022] PUT /identity/v3/projects/d4e559327d314c59b6345dae98803866/users/46e0ce6e66864773a04e81a796e637cb/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:06.872994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1e691-3f35-4de6-becf-5c08e876fe71 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:06.873208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1e691-3f35-4de6-becf-5c08e876fe71 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:06.873497 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1e691-3f35-4de6-becf-5c08e876fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:06.902831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-5de1e691-3f35-4de6-becf-5c08e876fe71 None None] MFA Rules not processed for user `46e0ce6e66864773a04e81a796e637cb`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:06.904080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-5de1e691-3f35-4de6-becf-5c08e876fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:06.946292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:06.969596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1272/2533] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:06.974208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820b793-b71e-4979-98be-b3a86692f237 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:07.037858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820b793-b71e-4979-98be-b3a86692f237 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:07.043295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e62090-936f-40ab-9138-866a96a1d7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:07.043901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e62090-936f-40ab-9138-866a96a1d7ca admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:07.044171 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e62090-936f-40ab-9138-866a96a1d7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:07.044437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e62090-936f-40ab-9138-866a96a1d7ca admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:07.045062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e62090-936f-40ab-9138-866a96a1d7ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:07.046945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e62090-936f-40ab-9138-866a96a1d7ca admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:07.185573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1262/2534] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:06 2022] POST /identity/v3/projects => generated 380 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:07.189761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1e691-3f35-4de6-becf-5c08e876fe71 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:07.201657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1e691-3f35-4de6-becf-5c08e876fe71 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:07.264101 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876463-2408-45ad-b32b-8c98169f30a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:07.264772 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876463-2408-45ad-b32b-8c98169f30a1 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:07.264996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876463-2408-45ad-b32b-8c98169f30a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:07.265262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876463-2408-45ad-b32b-8c98169f30a1 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:07.265890 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97876463-2408-45ad-b32b-8c98169f30a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:07.268035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97876463-2408-45ad-b32b-8c98169f30a1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:07.362035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1273/2535] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:07 2022] POST /identity/v3/users => generated 297 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:07.366840 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e62090-936f-40ab-9138-866a96a1d7ca admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:07.384086 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e62090-936f-40ab-9138-866a96a1d7ca admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:07.390370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeeeb47-b15c-4be7-a6c1-f92b5242c705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:07.391237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeeeb47-b15c-4be7-a6c1-f92b5242c705 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:07.391628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeeeb47-b15c-4be7-a6c1-f92b5242c705 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:07.391996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeeeb47-b15c-4be7-a6c1-f92b5242c705 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:07.392749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeeeb47-b15c-4be7-a6c1-f92b5242c705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:07.394629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeeeb47-b15c-4be7-a6c1-f92b5242c705 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:07.408822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1263/2536] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:07 2022] GET /identity/v3/roles => generated 1427 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:07.413850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876463-2408-45ad-b32b-8c98169f30a1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:07.425691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876463-2408-45ad-b32b-8c98169f30a1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:07.432359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66921a5a-4290-4937-a8d7-31d4a9447626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:07.433178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66921a5a-4290-4937-a8d7-31d4a9447626 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:07.433562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66921a5a-4290-4937-a8d7-31d4a9447626 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:07.434016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66921a5a-4290-4937-a8d7-31d4a9447626 admin admin] PATH_INFO: `/v3/projects/1bfb05cf76d345db9bcdb0c0a471433c/users/fbd029f357174584bba94b68db1cb41c/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:07.460596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66921a5a-4290-4937-a8d7-31d4a9447626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bfb05cf76d345db9bcdb0c0a471433c, user_id=fbd029f357174584bba94b68db1cb41c, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:07.465609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66921a5a-4290-4937-a8d7-31d4a9447626 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:07.559990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1274/2537] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:07 2022] PUT /identity/v3/projects/1bfb05cf76d345db9bcdb0c0a471433c/users/fbd029f357174584bba94b68db1cb41c/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:07.564268 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeeeb47-b15c-4be7-a6c1-f92b5242c705 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:07.577210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeeeb47-b15c-4be7-a6c1-f92b5242c705 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:07.652447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8e80a-d51f-4f9d-a068-60f9413e8846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:07.653065 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8e80a-d51f-4f9d-a068-60f9413e8846 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:07.653345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8e80a-d51f-4f9d-a068-60f9413e8846 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:07.653618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8e80a-d51f-4f9d-a068-60f9413e8846 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:07.654253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8e80a-d51f-4f9d-a068-60f9413e8846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:07.655895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8e80a-d51f-4f9d-a068-60f9413e8846 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:07.661522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:07.667646 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1264/2538] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:07 2022] GET /identity/v3/roles => generated 1427 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:11:07.671926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66921a5a-4290-4937-a8d7-31d4a9447626 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:07.683195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66921a5a-4290-4937-a8d7-31d4a9447626 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:07.687959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca78d0-0669-4fb7-a1ba-8fd2e65b8d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:07.688625 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca78d0-0669-4fb7-a1ba-8fd2e65b8d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:07.688907 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca78d0-0669-4fb7-a1ba-8fd2e65b8d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:07.691764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eca78d0-0669-4fb7-a1ba-8fd2e65b8d1b admin admin] PATH_INFO: `/v3/projects/1bfb05cf76d345db9bcdb0c0a471433c/users/fbd029f357174584bba94b68db1cb41c/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:07.691764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca78d0-0669-4fb7-a1ba-8fd2e65b8d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bfb05cf76d345db9bcdb0c0a471433c, user_id=fbd029f357174584bba94b68db1cb41c, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:07.693909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eca78d0-0669-4fb7-a1ba-8fd2e65b8d1b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:07.705240 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1275/2539] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:07 2022] PUT /identity/v3/projects/1bfb05cf76d345db9bcdb0c0a471433c/users/fbd029f357174584bba94b68db1cb41c/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:07.718352 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce20b7-8a42-435f-8c78-c306349be176 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:07.718572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce20b7-8a42-435f-8c78-c306349be176 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:07.718893 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ce20b7-8a42-435f-8c78-c306349be176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:07.750860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-94ce20b7-8a42-435f-8c78-c306349be176 None None] MFA Rules not processed for user `fbd029f357174584bba94b68db1cb41c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:07.753432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-94ce20b7-8a42-435f-8c78-c306349be176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:07.845858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1265/2540] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:07.857505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca78d0-0669-4fb7-a1ba-8fd2e65b8d1b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:07.859659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6eca78d0-0669-4fb7-a1ba-8fd2e65b8d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:07.923600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eca78d0-0669-4fb7-a1ba-8fd2e65b8d1b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:07.927616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d579d-1322-4e09-9c1b-4741fe9d104e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:07.928258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d579d-1322-4e09-9c1b-4741fe9d104e service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:07.928516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d579d-1322-4e09-9c1b-4741fe9d104e service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:07.928786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d579d-1322-4e09-9c1b-4741fe9d104e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:07.938827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d579d-1322-4e09-9c1b-4741fe9d104e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:07.940307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d579d-1322-4e09-9c1b-4741fe9d104e service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:07.953259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1276/2541] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:07 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:08.569893 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce20b7-8a42-435f-8c78-c306349be176 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:08.637207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ce20b7-8a42-435f-8c78-c306349be176 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:08.641826 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:08.642456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:08.642787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:08.642976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] PATH_INFO: `/v3/users/46e0ce6e66864773a04e81a796e637cb` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:08.644134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e0ce6e66864773a04e81a796e637cb)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:08.646047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:08.832455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e0ce6e66864773a04e81a796e637cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:08.833385 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e0ce6e66864773a04e81a796e637cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:08.833977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e0ce6e66864773a04e81a796e637cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:08.918531 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e0ce6e66864773a04e81a796e637cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:08.932785 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e0ce6e66864773a04e81a796e637cb'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:08.944359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1266/2542] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:08 2022] DELETE /identity/v3/users/46e0ce6e66864773a04e81a796e637cb => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:08.948955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026723d-d623-4b63-8943-7b1c7f575b0a None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:08.949451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026723d-d623-4b63-8943-7b1c7f575b0a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:08.949568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026723d-d623-4b63-8943-7b1c7f575b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:08.969408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-a026723d-d623-4b63-8943-7b1c7f575b0a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:08.970619 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a026723d-d623-4b63-8943-7b1c7f575b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:09.004492 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:09.032757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1277/2543] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:09.050940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:09.052823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:09.060144 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:09.110855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db181fa8-ae09-43b6-a693-8973546e42e5 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:09.114572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824178e-aacb-4da1-b2a4-2d6c5010cb32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:09.115236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824178e-aacb-4da1-b2a4-2d6c5010cb32 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:09.115518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824178e-aacb-4da1-b2a4-2d6c5010cb32 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:09.115791 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824178e-aacb-4da1-b2a4-2d6c5010cb32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:09.126428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824178e-aacb-4da1-b2a4-2d6c5010cb32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:09.128016 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824178e-aacb-4da1-b2a4-2d6c5010cb32 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:09.139423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1267/2544] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:09.919553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026723d-d623-4b63-8943-7b1c7f575b0a None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:09.921098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-a026723d-d623-4b63-8943-7b1c7f575b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:09.940812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026723d-d623-4b63-8943-7b1c7f575b0a None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:09.941457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:09.941827 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:09.942106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:09.942375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] PATH_INFO: `/v3/projects/d4e559327d314c59b6345dae98803866` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:09.943582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4e559327d314c59b6345dae98803866)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:09.945780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:10.071515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4e559327d314c59b6345dae98803866', 'request_id': 'req-8ca5775d-a799-4cf2-80a9-83260efe1b13'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:10.078243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4e559327d314c59b6345dae98803866', 'request_id': 'req-8ca5775d-a799-4cf2-80a9-83260efe1b13'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:10.084153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4e559327d314c59b6345dae98803866', 'request_id': 'req-8ca5775d-a799-4cf2-80a9-83260efe1b13'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:10.149859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] The token cache is being invalidate because project d4e559327d314c59b6345dae98803866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:10.150704 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:10.151599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:10.153185 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1278/2545] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:09 2022] DELETE /identity/v3/projects/d4e559327d314c59b6345dae98803866 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:10.157270 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824178e-aacb-4da1-b2a4-2d6c5010cb32 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:10.168106 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824178e-aacb-4da1-b2a4-2d6c5010cb32 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:10.235204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:10.235833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:10.236109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:10.236469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] PATH_INFO: `/v3/users/fbd029f357174584bba94b68db1cb41c` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:10.237644 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd029f357174584bba94b68db1cb41c)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:10.239521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:10.470477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd029f357174584bba94b68db1cb41c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:10.471164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd029f357174584bba94b68db1cb41c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:10.471781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd029f357174584bba94b68db1cb41c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:10.546968 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd029f357174584bba94b68db1cb41c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:10.559983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-9f91d32f-0413-457e-8138-2ea009a8a151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd029f357174584bba94b68db1cb41c'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:10.573216 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1268/2546] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:10 2022] DELETE /identity/v3/users/fbd029f357174584bba94b68db1cb41c => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:11.029245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:11.030754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:11.047670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5775d-a799-4cf2-80a9-83260efe1b13 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:11.126062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:11.126745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:11.127040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:11.127327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] PATH_INFO: `/v3/projects/1bfb05cf76d345db9bcdb0c0a471433c` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:11.128494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bfb05cf76d345db9bcdb0c0a471433c)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:11.130399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:11.245945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bfb05cf76d345db9bcdb0c0a471433c', 'request_id': 'req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:11.252290 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bfb05cf76d345db9bcdb0c0a471433c', 'request_id': 'req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:11.261370 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bfb05cf76d345db9bcdb0c0a471433c', 'request_id': 'req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:11.263472 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:11.380995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] The token cache is being invalidate because project 1bfb05cf76d345db9bcdb0c0a471433c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:11.381210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:11.381899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c1cc8477-2c02-4370-b0f5-19fdb1ec7a90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:11.383898 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1279/2547] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:11 2022] DELETE /identity/v3/projects/1bfb05cf76d345db9bcdb0c0a471433c => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:11.389319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f3aec-d1ab-4f92-990e-4043582f63d8 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:11.389941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f3aec-d1ab-4f92-990e-4043582f63d8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:11.390063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f3aec-d1ab-4f92-990e-4043582f63d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:11.412355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-885f3aec-d1ab-4f92-990e-4043582f63d8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:11.413589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-885f3aec-d1ab-4f92-990e-4043582f63d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:11.437928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:11.479426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1269/2548] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:11.487566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d13aae-ffcf-41ea-a6bb-7e9edccd4426 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:11.490757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d13aae-ffcf-41ea-a6bb-7e9edccd4426 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:11.490757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d13aae-ffcf-41ea-a6bb-7e9edccd4426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:11.516946 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-85d13aae-ffcf-41ea-a6bb-7e9edccd4426 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:11.519413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-85d13aae-ffcf-41ea-a6bb-7e9edccd4426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:11.528830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1280/2549] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:11.532890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f3aec-d1ab-4f92-990e-4043582f63d8 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:11.543079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f3aec-d1ab-4f92-990e-4043582f63d8 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:11.558723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101a236-fe28-4b0a-b24e-8c37006c9db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:11.559407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101a236-fe28-4b0a-b24e-8c37006c9db8 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:11.559679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101a236-fe28-4b0a-b24e-8c37006c9db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:11.559946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101a236-fe28-4b0a-b24e-8c37006c9db8 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:11.561471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101a236-fe28-4b0a-b24e-8c37006c9db8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:11.561471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101a236-fe28-4b0a-b24e-8c37006c9db8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:11.562781 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101a236-fe28-4b0a-b24e-8c37006c9db8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:11.577373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1270/2550] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:11:11.581947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d13aae-ffcf-41ea-a6bb-7e9edccd4426 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:11.593608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d13aae-ffcf-41ea-a6bb-7e9edccd4426 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:11.599281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cdebe-acaf-41d3-a149-24ff3ac6cb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:11.600074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cdebe-acaf-41d3-a149-24ff3ac6cb02 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:11.600522 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cdebe-acaf-41d3-a149-24ff3ac6cb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:11.600854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cdebe-acaf-41d3-a149-24ff3ac6cb02 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:11.601557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cdebe-acaf-41d3-a149-24ff3ac6cb02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:11.604046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cdebe-acaf-41d3-a149-24ff3ac6cb02 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:11.701371 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1281/2551] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:11 2022] POST /identity/v3/projects => generated 372 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:11.705589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101a236-fe28-4b0a-b24e-8c37006c9db8 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:11.716935 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101a236-fe28-4b0a-b24e-8c37006c9db8 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:11.781925 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3d3cf-3156-4724-a631-c09f87277501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:11.782656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d3cf-3156-4724-a631-c09f87277501 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:11.782940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d3cf-3156-4724-a631-c09f87277501 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:11.783242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3d3cf-3156-4724-a631-c09f87277501 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:11.783894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3d3cf-3156-4724-a631-c09f87277501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:11.785855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f3d3cf-3156-4724-a631-c09f87277501 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:11.899624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1271/2552] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:11 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:11.904729 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cdebe-acaf-41d3-a149-24ff3ac6cb02 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:11.915171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cdebe-acaf-41d3-a149-24ff3ac6cb02 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:11.920238 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a5b18-291b-49b0-a014-b8fa594c97bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:11.920835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a5b18-291b-49b0-a014-b8fa594c97bf admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:11.921114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a5b18-291b-49b0-a014-b8fa594c97bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:11.921379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a5b18-291b-49b0-a014-b8fa594c97bf admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:11.922105 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a5b18-291b-49b0-a014-b8fa594c97bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:11.923727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a5b18-291b-49b0-a014-b8fa594c97bf admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:11.934720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1282/2553] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:11 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:11.938912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3d3cf-3156-4724-a631-c09f87277501 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:11.949941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f3d3cf-3156-4724-a631-c09f87277501 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:11.954953 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41aeb0-e50b-4ad7-9450-30225668ffb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:11.955536 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41aeb0-e50b-4ad7-9450-30225668ffb8 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:11.955814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41aeb0-e50b-4ad7-9450-30225668ffb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:11.956155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41aeb0-e50b-4ad7-9450-30225668ffb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:11.956801 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41aeb0-e50b-4ad7-9450-30225668ffb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:11.958470 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41aeb0-e50b-4ad7-9450-30225668ffb8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:11.970806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1272/2554] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:11 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:11.975106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a5b18-291b-49b0-a014-b8fa594c97bf admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:11.985433 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a5b18-291b-49b0-a014-b8fa594c97bf admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:11.990110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c38af3-b014-4802-ac4f-78afc8dc7e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:11.990823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c38af3-b014-4802-ac4f-78afc8dc7e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:11.991282 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c38af3-b014-4802-ac4f-78afc8dc7e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:11.991655 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c38af3-b014-4802-ac4f-78afc8dc7e0a admin admin] PATH_INFO: `/v3/projects/335b48b79b384dd5bfde02ebe0c18a14/users/1e6d1027d2724a04b56b9110f02cb03a/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:12.008095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c38af3-b014-4802-ac4f-78afc8dc7e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=335b48b79b384dd5bfde02ebe0c18a14, user_id=1e6d1027d2724a04b56b9110f02cb03a, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:12.011340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c38af3-b014-4802-ac4f-78afc8dc7e0a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:12.094350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1283/2555] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:11 2022] PUT /identity/v3/projects/335b48b79b384dd5bfde02ebe0c18a14/users/1e6d1027d2724a04b56b9110f02cb03a/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:12.099654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40542d16-0c49-4c3a-b2c9-160ce5f33e66 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:12.099940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40542d16-0c49-4c3a-b2c9-160ce5f33e66 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:12.100213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40542d16-0c49-4c3a-b2c9-160ce5f33e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:12.124537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-40542d16-0c49-4c3a-b2c9-160ce5f33e66 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:12.125924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-40542d16-0c49-4c3a-b2c9-160ce5f33e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:12.189263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1273/2556] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:12.199444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c38af3-b014-4802-ac4f-78afc8dc7e0a admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:12.201171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-42c38af3-b014-4802-ac4f-78afc8dc7e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:12.260268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c38af3-b014-4802-ac4f-78afc8dc7e0a admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:12.263292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b08df-6ef8-45f2-9382-bc4c426ec622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:12.263923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b08df-6ef8-45f2-9382-bc4c426ec622 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:12.264201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b08df-6ef8-45f2-9382-bc4c426ec622 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:12.264471 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b08df-6ef8-45f2-9382-bc4c426ec622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:12.274109 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b08df-6ef8-45f2-9382-bc4c426ec622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:12.276155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b08df-6ef8-45f2-9382-bc4c426ec622 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:12.286381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1284/2557] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:13.457473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f14b8-5600-49ee-8373-38b4b410f90c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:13.458040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f14b8-5600-49ee-8373-38b4b410f90c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:13.458040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f14b8-5600-49ee-8373-38b4b410f90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:13.491079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5f2f14b8-5600-49ee-8373-38b4b410f90c None None] MFA Rules not processed for user `1e6d1027d2724a04b56b9110f02cb03a`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:13.492336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5f2f14b8-5600-49ee-8373-38b4b410f90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:13.517036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:13.580394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1274/2558] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:13.585279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b08df-6ef8-45f2-9382-bc4c426ec622 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:13.597478 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b08df-6ef8-45f2-9382-bc4c426ec622 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:13.602620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70daba5-a3d7-486c-bc6c-402b57615640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:13.603441 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70daba5-a3d7-486c-bc6c-402b57615640 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:13.603642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70daba5-a3d7-486c-bc6c-402b57615640 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:13.603910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70daba5-a3d7-486c-bc6c-402b57615640 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:13.604550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70daba5-a3d7-486c-bc6c-402b57615640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:13.606485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70daba5-a3d7-486c-bc6c-402b57615640 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:13.746887 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1285/2559] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:13 2022] POST /identity/v3/projects => generated 374 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:13.750846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f14b8-5600-49ee-8373-38b4b410f90c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:13.764096 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f14b8-5600-49ee-8373-38b4b410f90c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:13.859662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99fef7-a88e-441d-8fcf-d6c7492363c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:13.860568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99fef7-a88e-441d-8fcf-d6c7492363c9 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:13.860984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99fef7-a88e-441d-8fcf-d6c7492363c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:13.861390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99fef7-a88e-441d-8fcf-d6c7492363c9 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:13.862348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99fef7-a88e-441d-8fcf-d6c7492363c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:13.865535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e99fef7-a88e-441d-8fcf-d6c7492363c9 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:13.982647 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1275/2560] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:13 2022] POST /identity/v3/users => generated 294 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:13.988681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70daba5-a3d7-486c-bc6c-402b57615640 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:14.003266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70daba5-a3d7-486c-bc6c-402b57615640 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:14.011001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaaf61-ac9c-4838-8d68-80aacd457198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:14.011942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaaf61-ac9c-4838-8d68-80aacd457198 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:14.012367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaaf61-ac9c-4838-8d68-80aacd457198 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:14.012776 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaaf61-ac9c-4838-8d68-80aacd457198 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:14.013755 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaaf61-ac9c-4838-8d68-80aacd457198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:14.016410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaaf61-ac9c-4838-8d68-80aacd457198 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:14.032071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1286/2561] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:13 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:14.038279 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99fef7-a88e-441d-8fcf-d6c7492363c9 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:14.054690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99fef7-a88e-441d-8fcf-d6c7492363c9 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:14.062307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616fbe1-3850-4491-96d8-a1e1ed0ca9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:14.063394 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fbe1-3850-4491-96d8-a1e1ed0ca9f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:14.063815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fbe1-3850-4491-96d8-a1e1ed0ca9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:14.064262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fbe1-3850-4491-96d8-a1e1ed0ca9f5 admin admin] PATH_INFO: `/v3/projects/35bcca433a86411ca5c1a55285805c93/users/02a7a72e1d3a407bbd09e10a05ef60c0/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:14.090001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616fbe1-3850-4491-96d8-a1e1ed0ca9f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35bcca433a86411ca5c1a55285805c93, user_id=02a7a72e1d3a407bbd09e10a05ef60c0, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:14.094886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616fbe1-3850-4491-96d8-a1e1ed0ca9f5 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:14.179848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1276/2562] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:14 2022] PUT /identity/v3/projects/35bcca433a86411ca5c1a55285805c93/users/02a7a72e1d3a407bbd09e10a05ef60c0/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:14.183947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaaf61-ac9c-4838-8d68-80aacd457198 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:14.199849 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaaf61-ac9c-4838-8d68-80aacd457198 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:14.252001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:14.257735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7785c-3ebc-47ee-816a-2ec471e23749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:14.258340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7785c-3ebc-47ee-816a-2ec471e23749 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:14.258613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7785c-3ebc-47ee-816a-2ec471e23749 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:14.258884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7785c-3ebc-47ee-816a-2ec471e23749 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:14.260119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7785c-3ebc-47ee-816a-2ec471e23749 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:14.262408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7785c-3ebc-47ee-816a-2ec471e23749 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:14.271883 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1287/2563] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:14 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:14.277110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616fbe1-3850-4491-96d8-a1e1ed0ca9f5 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:14.287849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616fbe1-3850-4491-96d8-a1e1ed0ca9f5 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:14.293089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94e264-1aff-42dc-928c-3c8af5710bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:14.293749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94e264-1aff-42dc-928c-3c8af5710bdf admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:14.294028 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94e264-1aff-42dc-928c-3c8af5710bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:14.294300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94e264-1aff-42dc-928c-3c8af5710bdf admin admin] PATH_INFO: `/v3/projects/35bcca433a86411ca5c1a55285805c93/users/02a7a72e1d3a407bbd09e10a05ef60c0/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:14.296237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94e264-1aff-42dc-928c-3c8af5710bdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=35bcca433a86411ca5c1a55285805c93, user_id=02a7a72e1d3a407bbd09e10a05ef60c0, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:14.298884 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94e264-1aff-42dc-928c-3c8af5710bdf admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:14.309818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1277/2564] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:14 2022] PUT /identity/v3/projects/35bcca433a86411ca5c1a55285805c93/users/02a7a72e1d3a407bbd09e10a05ef60c0/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:15.627195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39f897-64f2-4923-bb32-5b85adb27b17 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:15.627195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39f897-64f2-4923-bb32-5b85adb27b17 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:15.627195 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39f897-64f2-4923-bb32-5b85adb27b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:15.654843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-dc39f897-64f2-4923-bb32-5b85adb27b17 None None] MFA Rules not processed for user `02a7a72e1d3a407bbd09e10a05ef60c0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:15.656132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-dc39f897-64f2-4923-bb32-5b85adb27b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:15.758207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1288/2565] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:15.768853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94e264-1aff-42dc-928c-3c8af5710bdf admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:15.770767 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9a94e264-1aff-42dc-928c-3c8af5710bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:15.837227 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94e264-1aff-42dc-928c-3c8af5710bdf admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:15.841416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d3fb0-e45e-40ee-94db-4df791f61006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:15.842036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d3fb0-e45e-40ee-94db-4df791f61006 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:15.842318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d3fb0-e45e-40ee-94db-4df791f61006 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:15.842595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d3fb0-e45e-40ee-94db-4df791f61006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:15.873484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:15.907388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d3fb0-e45e-40ee-94db-4df791f61006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:15.908831 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d3fb0-e45e-40ee-94db-4df791f61006 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:15.918682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1278/2566] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:15 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:18.746607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39f897-64f2-4923-bb32-5b85adb27b17 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:18.756384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39f897-64f2-4923-bb32-5b85adb27b17 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:18.763748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc09fbe-ebf7-46b4-abc9-a8aad5400b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:18.763748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc09fbe-ebf7-46b4-abc9-a8aad5400b7c service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:18.763748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc09fbe-ebf7-46b4-abc9-a8aad5400b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:18.763748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc09fbe-ebf7-46b4-abc9-a8aad5400b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:18.771458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc09fbe-ebf7-46b4-abc9-a8aad5400b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:18.772908 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc09fbe-ebf7-46b4-abc9-a8aad5400b7c service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:18.783858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1289/2567] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:25.984702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=14 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:28.779194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=43 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:36.672409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d3fb0-e45e-40ee-94db-4df791f61006 service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:36.673952 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-033d3fb0-e45e-40ee-94db-4df791f61006 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:36.754447 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d3fb0-e45e-40ee-94db-4df791f61006 service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:36.759549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:36.760233 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:36.760510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:36.760853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] PATH_INFO: `/v3/users/a8677a70f5ed4b96bf32018b9638fac2` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:36.762945 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8677a70f5ed4b96bf32018b9638fac2)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:36.765174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:37.006846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8677a70f5ed4b96bf32018b9638fac2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:37.007837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8677a70f5ed4b96bf32018b9638fac2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:37.009318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8677a70f5ed4b96bf32018b9638fac2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:37.084987 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8677a70f5ed4b96bf32018b9638fac2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:37.104545 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8677a70f5ed4b96bf32018b9638fac2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:37.121388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1279/2568] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:36 2022] DELETE /identity/v3/users/a8677a70f5ed4b96bf32018b9638fac2 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:37.125934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7baa6-f48d-482d-9428-e79a6574a001 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:37.126461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7baa6-f48d-482d-9428-e79a6574a001 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:37.126461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7baa6-f48d-482d-9428-e79a6574a001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:37.146892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-30d7baa6-f48d-482d-9428-e79a6574a001 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:37.148187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-30d7baa6-f48d-482d-9428-e79a6574a001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:37.225513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1290/2569] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:11:37.237044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:37.240578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:37.304698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6563d886-da9a-4f4e-a7e5-814643ab68b5 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:37.308503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca064fc7-bf9e-4675-81d2-dfc0a9e84ac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:37.309122 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca064fc7-bf9e-4675-81d2-dfc0a9e84ac4 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:37.309404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca064fc7-bf9e-4675-81d2-dfc0a9e84ac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:37.309676 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca064fc7-bf9e-4675-81d2-dfc0a9e84ac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:37.319346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca064fc7-bf9e-4675-81d2-dfc0a9e84ac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:37.320787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca064fc7-bf9e-4675-81d2-dfc0a9e84ac4 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:37.331320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1280/2570] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:37.812298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7baa6-f48d-482d-9428-e79a6574a001 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:37.813734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-30d7baa6-f48d-482d-9428-e79a6574a001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:37.826694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7baa6-f48d-482d-9428-e79a6574a001 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:37.832010 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:37.832656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:37.832936 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:37.833365 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] PATH_INFO: `/v3/projects/9e0bf8c9c5d44bed8b7b2bcd19d6c6c1` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:37.834523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e0bf8c9c5d44bed8b7b2bcd19d6c6c1)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:37.836736 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:37.948192 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e0bf8c9c5d44bed8b7b2bcd19d6c6c1', 'request_id': 'req-1d87e8a3-8afd-4f08-b987-d69bf2806de1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:37.955618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e0bf8c9c5d44bed8b7b2bcd19d6c6c1', 'request_id': 'req-1d87e8a3-8afd-4f08-b987-d69bf2806de1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:37.961726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e0bf8c9c5d44bed8b7b2bcd19d6c6c1', 'request_id': 'req-1d87e8a3-8afd-4f08-b987-d69bf2806de1'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:38.128773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] The token cache is being invalidate because project 9e0bf8c9c5d44bed8b7b2bcd19d6c6c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:38.129451 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:38.129567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:38.131100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1291/2571] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:37 2022] DELETE /identity/v3/projects/9e0bf8c9c5d44bed8b7b2bcd19d6c6c1 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:38.135187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca064fc7-bf9e-4675-81d2-dfc0a9e84ac4 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:38.145750 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca064fc7-bf9e-4675-81d2-dfc0a9e84ac4 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:38.164211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:38.207291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:38.207954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:38.208232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:38.208505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] PATH_INFO: `/v3/users/5bfe1c81cf294d9a956f54c238fb6578` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:38.209714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bfe1c81cf294d9a956f54c238fb6578)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:38.211711 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:38.397485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bfe1c81cf294d9a956f54c238fb6578'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:38.398055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bfe1c81cf294d9a956f54c238fb6578'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:38.398667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bfe1c81cf294d9a956f54c238fb6578'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:38.482726 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bfe1c81cf294d9a956f54c238fb6578'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:38.496842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7297bd9b-a00d-4354-bc00-fb216a9caf7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bfe1c81cf294d9a956f54c238fb6578'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:38.510002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1281/2572] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:38 2022] DELETE /identity/v3/users/5bfe1c81cf294d9a956f54c238fb6578 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:39.010548 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:39.018622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:39.033459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e8a3-8afd-4f08-b987-d69bf2806de1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:39.108900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:39.109683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:39.110009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:39.110392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] PATH_INFO: `/v3/projects/19f97cb9ab4b4593ae04ca27b1759cf4` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:39.111701 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19f97cb9ab4b4593ae04ca27b1759cf4)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:39.113748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:39.232818 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:39.251790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19f97cb9ab4b4593ae04ca27b1759cf4', 'request_id': 'req-663d4b58-ba89-402d-830b-84a2e3001201'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:39.258337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19f97cb9ab4b4593ae04ca27b1759cf4', 'request_id': 'req-663d4b58-ba89-402d-830b-84a2e3001201'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:39.265353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19f97cb9ab4b4593ae04ca27b1759cf4', 'request_id': 'req-663d4b58-ba89-402d-830b-84a2e3001201'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:39.350809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] The token cache is being invalidate because project 19f97cb9ab4b4593ae04ca27b1759cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:39.351191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:39.351774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-663d4b58-ba89-402d-830b-84a2e3001201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:39.353306 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1292/2573] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:39 2022] DELETE /identity/v3/projects/19f97cb9ab4b4593ae04ca27b1759cf4 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:39.359336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38130d-639b-4e6f-a725-8e0b14efff98 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:39.359956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38130d-639b-4e6f-a725-8e0b14efff98 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:39.359956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38130d-639b-4e6f-a725-8e0b14efff98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:39.380022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-cd38130d-639b-4e6f-a725-8e0b14efff98 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:39.381266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-cd38130d-639b-4e6f-a725-8e0b14efff98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:39.443550 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1282/2574] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:39.452466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30ec34-ebba-4b0a-8751-17185be288c2 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:39.452779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30ec34-ebba-4b0a-8751-17185be288c2 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:39.453055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30ec34-ebba-4b0a-8751-17185be288c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:39.474431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-3a30ec34-ebba-4b0a-8751-17185be288c2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:39.475693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3a30ec34-ebba-4b0a-8751-17185be288c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:39.484221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1293/2575] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:39.491535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38130d-639b-4e6f-a725-8e0b14efff98 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:39.501374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38130d-639b-4e6f-a725-8e0b14efff98 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:39.505899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03e16a-92ae-406b-bdd5-92e460ea2547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:39.506533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03e16a-92ae-406b-bdd5-92e460ea2547 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:39.506805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03e16a-92ae-406b-bdd5-92e460ea2547 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:39.507249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03e16a-92ae-406b-bdd5-92e460ea2547 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:39.507903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03e16a-92ae-406b-bdd5-92e460ea2547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:39.508207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03e16a-92ae-406b-bdd5-92e460ea2547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:39.510484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03e16a-92ae-406b-bdd5-92e460ea2547 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:39.523832 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1283/2576] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:11:39.528943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30ec34-ebba-4b0a-8751-17185be288c2 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:39.538321 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30ec34-ebba-4b0a-8751-17185be288c2 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:39.543266 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd89e3-8fe6-475d-b506-6e6c5442a65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:39.543878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd89e3-8fe6-475d-b506-6e6c5442a65d admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:39.544202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd89e3-8fe6-475d-b506-6e6c5442a65d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:39.544443 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd89e3-8fe6-475d-b506-6e6c5442a65d admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:39.545087 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd89e3-8fe6-475d-b506-6e6c5442a65d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:39.547320 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd89e3-8fe6-475d-b506-6e6c5442a65d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:39.647066 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1294/2577] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:39 2022] POST /identity/v3/projects => generated 388 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:11:39.651499 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03e16a-92ae-406b-bdd5-92e460ea2547 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:39.662542 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03e16a-92ae-406b-bdd5-92e460ea2547 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:39.691707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:39.726022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f409d-2599-4d21-b691-c377ce056b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:39.726636 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f409d-2599-4d21-b691-c377ce056b63 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:39.726913 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f409d-2599-4d21-b691-c377ce056b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:39.727234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156f409d-2599-4d21-b691-c377ce056b63 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:39.727977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f409d-2599-4d21-b691-c377ce056b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:39.729906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156f409d-2599-4d21-b691-c377ce056b63 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:39.829979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1284/2578] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:39 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:39.833654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd89e3-8fe6-475d-b506-6e6c5442a65d admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:39.844693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd89e3-8fe6-475d-b506-6e6c5442a65d admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:39.849422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7160e-8c2c-482d-9b28-45b591dd8d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:39.850050 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7160e-8c2c-482d-9b28-45b591dd8d8c admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:39.850324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7160e-8c2c-482d-9b28-45b591dd8d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:39.850591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e7160e-8c2c-482d-9b28-45b591dd8d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:39.851259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7160e-8c2c-482d-9b28-45b591dd8d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:39.852843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e7160e-8c2c-482d-9b28-45b591dd8d8c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:39.863963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1295/2579] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:39 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:39.868135 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f409d-2599-4d21-b691-c377ce056b63 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:39.879365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156f409d-2599-4d21-b691-c377ce056b63 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:39.884389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71c9f1-1cf2-4264-8582-b86d2f36cad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:39.885004 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71c9f1-1cf2-4264-8582-b86d2f36cad7 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:39.885283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71c9f1-1cf2-4264-8582-b86d2f36cad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:39.885555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71c9f1-1cf2-4264-8582-b86d2f36cad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:39.886196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71c9f1-1cf2-4264-8582-b86d2f36cad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:39.888413 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e71c9f1-1cf2-4264-8582-b86d2f36cad7 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:39.899330 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1285/2580] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:39 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:11:39.903668 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7160e-8c2c-482d-9b28-45b591dd8d8c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:39.914544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e7160e-8c2c-482d-9b28-45b591dd8d8c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:39.920357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8d899-f497-4b2d-827d-7052ff15f64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:39.921027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8d899-f497-4b2d-827d-7052ff15f64e admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:39.921308 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8d899-f497-4b2d-827d-7052ff15f64e admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:39.921580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8d899-f497-4b2d-827d-7052ff15f64e admin admin] PATH_INFO: `/v3/projects/6413eaa7e1e64f1089fde48b260ef755/users/78688c74856a44828014ecbd88240f92/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:39.939133 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8d899-f497-4b2d-827d-7052ff15f64e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6413eaa7e1e64f1089fde48b260ef755, user_id=78688c74856a44828014ecbd88240f92, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:39.942307 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8d899-f497-4b2d-827d-7052ff15f64e admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:40.122025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1296/2581] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:39 2022] PUT /identity/v3/projects/6413eaa7e1e64f1089fde48b260ef755/users/78688c74856a44828014ecbd88240f92/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:40.132732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319f085-752b-4260-bb19-6d486a1073c8 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:40.133022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319f085-752b-4260-bb19-6d486a1073c8 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:40.133301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319f085-752b-4260-bb19-6d486a1073c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:40.161653 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-e319f085-752b-4260-bb19-6d486a1073c8 None None] MFA Rules not processed for user `78688c74856a44828014ecbd88240f92`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:40.162856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e319f085-752b-4260-bb19-6d486a1073c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:40.233989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1286/2582] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:40 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:40.238930 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8d899-f497-4b2d-827d-7052ff15f64e admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:40.305765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8d899-f497-4b2d-827d-7052ff15f64e admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:40.311937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ca8c6-7a00-4326-924f-397ffc568583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:40.312966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ca8c6-7a00-4326-924f-397ffc568583 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:40.312966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ca8c6-7a00-4326-924f-397ffc568583 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:40.313178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ca8c6-7a00-4326-924f-397ffc568583 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:40.313759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ca8c6-7a00-4326-924f-397ffc568583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:40.315759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ca8c6-7a00-4326-924f-397ffc568583 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:40.562549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1297/2583] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:40 2022] POST /identity/v3/projects => generated 386 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:40.567890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319f085-752b-4260-bb19-6d486a1073c8 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:40.580806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319f085-752b-4260-bb19-6d486a1073c8 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:40.649561 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca68e5-151d-482c-bfea-05f2fa26bb8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:40.650181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca68e5-151d-482c-bfea-05f2fa26bb8e admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:40.650486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca68e5-151d-482c-bfea-05f2fa26bb8e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:40.650763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca68e5-151d-482c-bfea-05f2fa26bb8e admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:40.651510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca68e5-151d-482c-bfea-05f2fa26bb8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:40.653389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca68e5-151d-482c-bfea-05f2fa26bb8e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:40.757372 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1287/2584] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:40 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:40.761555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ca8c6-7a00-4326-924f-397ffc568583 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:40.772705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ca8c6-7a00-4326-924f-397ffc568583 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:40.777511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08353e-3632-4df6-9026-d0c7c3f61815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:40.778115 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08353e-3632-4df6-9026-d0c7c3f61815 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:40.778395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08353e-3632-4df6-9026-d0c7c3f61815 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:40.778675 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08353e-3632-4df6-9026-d0c7c3f61815 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:40.779357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08353e-3632-4df6-9026-d0c7c3f61815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:40.780974 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08353e-3632-4df6-9026-d0c7c3f61815 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:40.792034 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1298/2585] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:40 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:40.806261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca68e5-151d-482c-bfea-05f2fa26bb8e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:40.817374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca68e5-151d-482c-bfea-05f2fa26bb8e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:40.822112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a9e2e-114a-4221-bf3a-d985e38f2e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:40.822762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a9e2e-114a-4221-bf3a-d985e38f2e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:40.823061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a9e2e-114a-4221-bf3a-d985e38f2e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:40.823338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a9e2e-114a-4221-bf3a-d985e38f2e80 admin admin] PATH_INFO: `/v3/projects/4b4fc46c9cee4c1f8f9d1b832a2835c7/users/a1a8b2a5869e46a4bc407830d4a0a1b0/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:40.840296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a9e2e-114a-4221-bf3a-d985e38f2e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4fc46c9cee4c1f8f9d1b832a2835c7, user_id=a1a8b2a5869e46a4bc407830d4a0a1b0, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:40.843089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a9e2e-114a-4221-bf3a-d985e38f2e80 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:40.928135 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1288/2586] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:40 2022] PUT /identity/v3/projects/4b4fc46c9cee4c1f8f9d1b832a2835c7/users/a1a8b2a5869e46a4bc407830d4a0a1b0/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:40.932241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08353e-3632-4df6-9026-d0c7c3f61815 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:40.943694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08353e-3632-4df6-9026-d0c7c3f61815 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:40.965265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:41.004100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1844ae-e228-4e60-b9b6-03f90fb24bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:41.004715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1844ae-e228-4e60-b9b6-03f90fb24bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:41.005077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1844ae-e228-4e60-b9b6-03f90fb24bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:41.005350 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1844ae-e228-4e60-b9b6-03f90fb24bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:41.006049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1844ae-e228-4e60-b9b6-03f90fb24bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:41.007689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1844ae-e228-4e60-b9b6-03f90fb24bd6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:41.018503 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1299/2587] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:40 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:41.022763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a9e2e-114a-4221-bf3a-d985e38f2e80 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:41.029620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:41.035094 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a9e2e-114a-4221-bf3a-d985e38f2e80 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:41.040128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f243d-14d7-4609-a4c2-cc32da00b9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:41.040817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f243d-14d7-4609-a4c2-cc32da00b9ed admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:41.041334 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f243d-14d7-4609-a4c2-cc32da00b9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:41.041657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f243d-14d7-4609-a4c2-cc32da00b9ed admin admin] PATH_INFO: `/v3/projects/4b4fc46c9cee4c1f8f9d1b832a2835c7/users/a1a8b2a5869e46a4bc407830d4a0a1b0/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:41.044744 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f243d-14d7-4609-a4c2-cc32da00b9ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4fc46c9cee4c1f8f9d1b832a2835c7, user_id=a1a8b2a5869e46a4bc407830d4a0a1b0, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:41.047610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f243d-14d7-4609-a4c2-cc32da00b9ed admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:41.059091 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1289/2588] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:41 2022] PUT /identity/v3/projects/4b4fc46c9cee4c1f8f9d1b832a2835c7/users/a1a8b2a5869e46a4bc407830d4a0a1b0/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:41.074593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d4092-5641-4562-b6f1-d6dda552bcbc None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:41.074593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d4092-5641-4562-b6f1-d6dda552bcbc None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:41.074593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d4092-5641-4562-b6f1-d6dda552bcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:41.102882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-991d4092-5641-4562-b6f1-d6dda552bcbc None None] MFA Rules not processed for user `a1a8b2a5869e46a4bc407830d4a0a1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:41.104144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-991d4092-5641-4562-b6f1-d6dda552bcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:41.192850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1300/2589] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:41 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:41.205377 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f243d-14d7-4609-a4c2-cc32da00b9ed admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:41.207141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-fe6f243d-14d7-4609-a4c2-cc32da00b9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:41.272664 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f243d-14d7-4609-a4c2-cc32da00b9ed admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:41.277252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402be46b-1cb6-4197-9256-45226e8b724a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:41.279076 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402be46b-1cb6-4197-9256-45226e8b724a service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:41.279356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402be46b-1cb6-4197-9256-45226e8b724a service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:41.279633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402be46b-1cb6-4197-9256-45226e8b724a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:41.290309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402be46b-1cb6-4197-9256-45226e8b724a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:41.291892 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402be46b-1cb6-4197-9256-45226e8b724a service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:41.303294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1290/2590] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:41 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:41.396424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d4092-5641-4562-b6f1-d6dda552bcbc None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:41.473046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d4092-5641-4562-b6f1-d6dda552bcbc None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:41.477645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:41.478281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:41.478559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:41.478831 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] PATH_INFO: `/v3/users/78688c74856a44828014ecbd88240f92` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:41.480024 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78688c74856a44828014ecbd88240f92)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:41.482352 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:41.782403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78688c74856a44828014ecbd88240f92'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:41.783134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78688c74856a44828014ecbd88240f92'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:41.783775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78688c74856a44828014ecbd88240f92'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:41.785605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:41.940811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78688c74856a44828014ecbd88240f92'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:41.953895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78688c74856a44828014ecbd88240f92'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:41.967940 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1301/2591] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:41 2022] DELETE /identity/v3/users/78688c74856a44828014ecbd88240f92 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:41.972455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114c02c-2c2d-4056-9f6e-8563cffd0fea None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:41.972960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114c02c-2c2d-4056-9f6e-8563cffd0fea None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:41.972960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8114c02c-2c2d-4056-9f6e-8563cffd0fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:41.994639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8114c02c-2c2d-4056-9f6e-8563cffd0fea None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:41.994639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8114c02c-2c2d-4056-9f6e-8563cffd0fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:42.063192 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1291/2592] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:42.073218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:42.074866 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:42.134844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2185a89-17d1-460c-ae40-7284a68cce59 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:42.141507 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85645d8-a3cb-41a6-bf04-76847be6375d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:42.142122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85645d8-a3cb-41a6-bf04-76847be6375d service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:42.142399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85645d8-a3cb-41a6-bf04-76847be6375d service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:42.142667 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85645d8-a3cb-41a6-bf04-76847be6375d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:42.153622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85645d8-a3cb-41a6-bf04-76847be6375d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:42.155083 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85645d8-a3cb-41a6-bf04-76847be6375d service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:42.166179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1302/2593] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:42.647157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114c02c-2c2d-4056-9f6e-8563cffd0fea None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:42.648975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8114c02c-2c2d-4056-9f6e-8563cffd0fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:42.668301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8114c02c-2c2d-4056-9f6e-8563cffd0fea None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:42.680858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:42.680858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:42.680858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:42.680858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] PATH_INFO: `/v3/projects/6413eaa7e1e64f1089fde48b260ef755` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:42.681608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6413eaa7e1e64f1089fde48b260ef755)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:42.681608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:42.849002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6413eaa7e1e64f1089fde48b260ef755', 'request_id': 'req-c3055fbe-d84a-459a-9e74-7c7c3ee86124'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:42.856165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6413eaa7e1e64f1089fde48b260ef755', 'request_id': 'req-c3055fbe-d84a-459a-9e74-7c7c3ee86124'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:42.940489 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6413eaa7e1e64f1089fde48b260ef755', 'request_id': 'req-c3055fbe-d84a-459a-9e74-7c7c3ee86124'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:42.953694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] The token cache is being invalidate because project 6413eaa7e1e64f1089fde48b260ef755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:42.954011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:42.954680 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:42.956283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1292/2594] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:42 2022] DELETE /identity/v3/projects/6413eaa7e1e64f1089fde48b260ef755 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:42.960312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85645d8-a3cb-41a6-bf04-76847be6375d service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:42.970696 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85645d8-a3cb-41a6-bf04-76847be6375d service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:43.034747 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:43.035510 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:43.035797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:43.036070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] PATH_INFO: `/v3/users/a1a8b2a5869e46a4bc407830d4a0a1b0` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:43.037206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1a8b2a5869e46a4bc407830d4a0a1b0)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:43.039142 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:43.220042 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1a8b2a5869e46a4bc407830d4a0a1b0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:43.220854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1a8b2a5869e46a4bc407830d4a0a1b0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:43.221576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1a8b2a5869e46a4bc407830d4a0a1b0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:43.306205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1a8b2a5869e46a4bc407830d4a0a1b0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:43.319528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:43.320750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-a347c6ed-2ba7-407f-8674-74de8e065b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1a8b2a5869e46a4bc407830d4a0a1b0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:43.333862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1303/2595] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:42 2022] DELETE /identity/v3/users/a1a8b2a5869e46a4bc407830d4a0a1b0 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:43.842048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:43.843809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:43.861401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3055fbe-d84a-459a-9e74-7c7c3ee86124 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:43.876228 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:43.923097 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:43.923934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:43.924214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:43.924486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] PATH_INFO: `/v3/projects/4b4fc46c9cee4c1f8f9d1b832a2835c7` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:43.925647 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b4fc46c9cee4c1f8f9d1b832a2835c7)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:43.927857 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:44.083779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b4fc46c9cee4c1f8f9d1b832a2835c7', 'request_id': 'req-50283fd8-4f1a-4e85-874e-21ead02fa55e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:44.091294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b4fc46c9cee4c1f8f9d1b832a2835c7', 'request_id': 'req-50283fd8-4f1a-4e85-874e-21ead02fa55e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:44.197614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b4fc46c9cee4c1f8f9d1b832a2835c7', 'request_id': 'req-50283fd8-4f1a-4e85-874e-21ead02fa55e'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:44.212516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] The token cache is being invalidate because project 4b4fc46c9cee4c1f8f9d1b832a2835c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:44.212516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:44.213828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-50283fd8-4f1a-4e85-874e-21ead02fa55e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:44.216048 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1293/2596] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:43 2022] DELETE /identity/v3/projects/4b4fc46c9cee4c1f8f9d1b832a2835c7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:44.221147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a1c0c-94fb-4ddd-8a80-6e24459850de None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:44.221615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a1c0c-94fb-4ddd-8a80-6e24459850de None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:44.229222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a1c0c-94fb-4ddd-8a80-6e24459850de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:44.251450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-878a1c0c-94fb-4ddd-8a80-6e24459850de None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:44.252762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-878a1c0c-94fb-4ddd-8a80-6e24459850de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:44.326479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1304/2597] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:44.334464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3ac39-5cdd-477f-b6f6-600ac8680bc5 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:44.334778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3ac39-5cdd-477f-b6f6-600ac8680bc5 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:44.335086 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3ac39-5cdd-477f-b6f6-600ac8680bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:44.355994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-fea3ac39-5cdd-477f-b6f6-600ac8680bc5 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:44.357287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-fea3ac39-5cdd-477f-b6f6-600ac8680bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:44.365433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1294/2598] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:44.369780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a1c0c-94fb-4ddd-8a80-6e24459850de None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:44.380173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a1c0c-94fb-4ddd-8a80-6e24459850de None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:44.385358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce551-6282-4a41-9a18-8c7378abc9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:44.385762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce551-6282-4a41-9a18-8c7378abc9cf admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:44.386037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce551-6282-4a41-9a18-8c7378abc9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:44.386301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ce551-6282-4a41-9a18-8c7378abc9cf admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:44.387606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ce551-6282-4a41-9a18-8c7378abc9cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:44.387952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ce551-6282-4a41-9a18-8c7378abc9cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:44.389556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ce551-6282-4a41-9a18-8c7378abc9cf admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:44.404029 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1305/2599] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:11:44.408503 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3ac39-5cdd-477f-b6f6-600ac8680bc5 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:44.419293 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3ac39-5cdd-477f-b6f6-600ac8680bc5 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:44.425137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663c9ea-e179-44be-9d92-f15488424f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:44.425755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663c9ea-e179-44be-9d92-f15488424f17 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:44.426044 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663c9ea-e179-44be-9d92-f15488424f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:44.426347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663c9ea-e179-44be-9d92-f15488424f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:44.427042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663c9ea-e179-44be-9d92-f15488424f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:44.429189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663c9ea-e179-44be-9d92-f15488424f17 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:44.546342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1295/2600] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:44 2022] POST /identity/v3/projects => generated 382 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:44.551593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce551-6282-4a41-9a18-8c7378abc9cf admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:44.567774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ce551-6282-4a41-9a18-8c7378abc9cf admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:44.629634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102e90d-b170-4a8b-9119-6a2161312729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:44.630409 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102e90d-b170-4a8b-9119-6a2161312729 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:44.630686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102e90d-b170-4a8b-9119-6a2161312729 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:44.631286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102e90d-b170-4a8b-9119-6a2161312729 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:44.631919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102e90d-b170-4a8b-9119-6a2161312729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:44.633829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102e90d-b170-4a8b-9119-6a2161312729 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:44.749221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1306/2601] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:44 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:44.753307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663c9ea-e179-44be-9d92-f15488424f17 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:44.764973 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663c9ea-e179-44be-9d92-f15488424f17 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:44.770149 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374fd266-d11a-40bf-b698-f37e65fde686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:44.770786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fd266-d11a-40bf-b698-f37e65fde686 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:44.771219 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fd266-d11a-40bf-b698-f37e65fde686 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:44.771393 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fd266-d11a-40bf-b698-f37e65fde686 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:44.772025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374fd266-d11a-40bf-b698-f37e65fde686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:44.773730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374fd266-d11a-40bf-b698-f37e65fde686 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:44.784696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1296/2602] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:44 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:44.788867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102e90d-b170-4a8b-9119-6a2161312729 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:44.799963 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102e90d-b170-4a8b-9119-6a2161312729 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:44.804900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5ee72-7777-48c7-a5ef-a2939edb74cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:44.805490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5ee72-7777-48c7-a5ef-a2939edb74cc admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:44.805760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5ee72-7777-48c7-a5ef-a2939edb74cc admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:44.806025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5ee72-7777-48c7-a5ef-a2939edb74cc admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:44.806658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5ee72-7777-48c7-a5ef-a2939edb74cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:44.808456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5ee72-7777-48c7-a5ef-a2939edb74cc admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:44.820591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1307/2603] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:44 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:44.825221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374fd266-d11a-40bf-b698-f37e65fde686 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:44.836823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374fd266-d11a-40bf-b698-f37e65fde686 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:44.842287 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5b2e2-9106-45f7-80bb-1ea1d4c49fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:44.842922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5b2e2-9106-45f7-80bb-1ea1d4c49fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:44.843226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5b2e2-9106-45f7-80bb-1ea1d4c49fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:44.843560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5b2e2-9106-45f7-80bb-1ea1d4c49fd4 admin admin] PATH_INFO: `/v3/projects/e68f9553fd4945f0bc7e8152a298b53e/users/d1d5192fc6024c4182ef9a29f675e6ca/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:44.860387 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5b2e2-9106-45f7-80bb-1ea1d4c49fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e68f9553fd4945f0bc7e8152a298b53e, user_id=d1d5192fc6024c4182ef9a29f675e6ca, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:44.863050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5b2e2-9106-45f7-80bb-1ea1d4c49fd4 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:44.957569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1297/2604] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:44 2022] PUT /identity/v3/projects/e68f9553fd4945f0bc7e8152a298b53e/users/d1d5192fc6024c4182ef9a29f675e6ca/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:44.968522 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e7085-7344-4a62-b7e4-ca92332a4444 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:44.968809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e7085-7344-4a62-b7e4-ca92332a4444 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:44.969086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e7085-7344-4a62-b7e4-ca92332a4444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:44.998272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-613e7085-7344-4a62-b7e4-ca92332a4444 None None] MFA Rules not processed for user `d1d5192fc6024c4182ef9a29f675e6ca`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:44.999535 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-613e7085-7344-4a62-b7e4-ca92332a4444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:45.041138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:45.065015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1308/2605] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:45.069765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5b2e2-9106-45f7-80bb-1ea1d4c49fd4 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:45.114421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:45.133710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5b2e2-9106-45f7-80bb-1ea1d4c49fd4 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:45.138909 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ab53-de49-4b6c-a4f0-b68beee3f8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:45.139588 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ab53-de49-4b6c-a4f0-b68beee3f8a6 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:45.139871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ab53-de49-4b6c-a4f0-b68beee3f8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:45.140174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc9ab53-de49-4b6c-a4f0-b68beee3f8a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:45.140854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9ab53-de49-4b6c-a4f0-b68beee3f8a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:45.142784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc9ab53-de49-4b6c-a4f0-b68beee3f8a6 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:45.251655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1298/2606] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:45 2022] POST /identity/v3/projects => generated 382 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:45.255926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e7085-7344-4a62-b7e4-ca92332a4444 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:45.267206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e7085-7344-4a62-b7e4-ca92332a4444 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:45.328508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07641d94-3fc9-4b76-b500-515c126af107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:45.329123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07641d94-3fc9-4b76-b500-515c126af107 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:45.329404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07641d94-3fc9-4b76-b500-515c126af107 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:45.329720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07641d94-3fc9-4b76-b500-515c126af107 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:45.330477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07641d94-3fc9-4b76-b500-515c126af107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:45.332470 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07641d94-3fc9-4b76-b500-515c126af107 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:45.427354 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1309/2607] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:45 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:45.431615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ab53-de49-4b6c-a4f0-b68beee3f8a6 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:45.442871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc9ab53-de49-4b6c-a4f0-b68beee3f8a6 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:45.447724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa46ed4-a83d-4b24-80fa-7e9ff5e107a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:45.448344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa46ed4-a83d-4b24-80fa-7e9ff5e107a1 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:45.448623 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa46ed4-a83d-4b24-80fa-7e9ff5e107a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:45.448894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa46ed4-a83d-4b24-80fa-7e9ff5e107a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:45.450351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa46ed4-a83d-4b24-80fa-7e9ff5e107a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:45.451991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa46ed4-a83d-4b24-80fa-7e9ff5e107a1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:45.462531 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1299/2608] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:45 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:45.467781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07641d94-3fc9-4b76-b500-515c126af107 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:45.485501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07641d94-3fc9-4b76-b500-515c126af107 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:45.490804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9de878-dca7-4712-ba43-0313462ab9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:45.491509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9de878-dca7-4712-ba43-0313462ab9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:45.491791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9de878-dca7-4712-ba43-0313462ab9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:45.492074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9de878-dca7-4712-ba43-0313462ab9e1 admin admin] PATH_INFO: `/v3/projects/5f4c6c6fb9aa4d8bbccb659e2927fe55/users/6e4a8fa357a74064b1bafd7fc68b34d6/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:45.508923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9de878-dca7-4712-ba43-0313462ab9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f4c6c6fb9aa4d8bbccb659e2927fe55, user_id=6e4a8fa357a74064b1bafd7fc68b34d6, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:45.511674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9de878-dca7-4712-ba43-0313462ab9e1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:45.592792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1310/2609] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:45 2022] PUT /identity/v3/projects/5f4c6c6fb9aa4d8bbccb659e2927fe55/users/6e4a8fa357a74064b1bafd7fc68b34d6/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:45.599431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa46ed4-a83d-4b24-80fa-7e9ff5e107a1 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:45.612160 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa46ed4-a83d-4b24-80fa-7e9ff5e107a1 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:45.678062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8f0b9-f4c0-4ed8-ac30-92bda2401d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:45.678698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8f0b9-f4c0-4ed8-ac30-92bda2401d89 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:45.678960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8f0b9-f4c0-4ed8-ac30-92bda2401d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:45.679256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8f0b9-f4c0-4ed8-ac30-92bda2401d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:45.679890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8f0b9-f4c0-4ed8-ac30-92bda2401d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:45.681551 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8f0b9-f4c0-4ed8-ac30-92bda2401d89 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:45.694079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1300/2610] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:45 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:45.700237 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9de878-dca7-4712-ba43-0313462ab9e1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:45.711757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9de878-dca7-4712-ba43-0313462ab9e1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:45.717809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db712c0b-0e36-4c63-8a44-1322bafb16ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:45.718650 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db712c0b-0e36-4c63-8a44-1322bafb16ab admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:45.718995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db712c0b-0e36-4c63-8a44-1322bafb16ab admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:45.719368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db712c0b-0e36-4c63-8a44-1322bafb16ab admin admin] PATH_INFO: `/v3/projects/5f4c6c6fb9aa4d8bbccb659e2927fe55/users/6e4a8fa357a74064b1bafd7fc68b34d6/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:45.721429 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db712c0b-0e36-4c63-8a44-1322bafb16ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f4c6c6fb9aa4d8bbccb659e2927fe55, user_id=6e4a8fa357a74064b1bafd7fc68b34d6, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:45.725126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db712c0b-0e36-4c63-8a44-1322bafb16ab admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:45.742014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1311/2611] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:45 2022] PUT /identity/v3/projects/5f4c6c6fb9aa4d8bbccb659e2927fe55/users/6e4a8fa357a74064b1bafd7fc68b34d6/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:45.753830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8d53a-65d8-4bba-9345-de71b8ea6751 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:45.754198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8d53a-65d8-4bba-9345-de71b8ea6751 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:45.754383 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca8d53a-65d8-4bba-9345-de71b8ea6751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:45.798360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-bca8d53a-65d8-4bba-9345-de71b8ea6751 None None] MFA Rules not processed for user `6e4a8fa357a74064b1bafd7fc68b34d6`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:45.799622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-bca8d53a-65d8-4bba-9345-de71b8ea6751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:45.874728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1301/2612] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:45 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:45.888383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db712c0b-0e36-4c63-8a44-1322bafb16ab admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:45.890768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-db712c0b-0e36-4c63-8a44-1322bafb16ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:45.959252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db712c0b-0e36-4c63-8a44-1322bafb16ab admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:45.963410 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16119bc7-4b81-4491-8e12-3b4458fbf75a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:45.963978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16119bc7-4b81-4491-8e12-3b4458fbf75a service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:45.964252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16119bc7-4b81-4491-8e12-3b4458fbf75a service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:45.964523 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16119bc7-4b81-4491-8e12-3b4458fbf75a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:45.982352 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16119bc7-4b81-4491-8e12-3b4458fbf75a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:45.983947 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16119bc7-4b81-4491-8e12-3b4458fbf75a service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:45.994682 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1312/2613] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:45 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:47.247887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8d53a-65d8-4bba-9345-de71b8ea6751 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:47.247887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca8d53a-65d8-4bba-9345-de71b8ea6751 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:47.247887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a074e8-1af6-42f0-9471-af46b2d24d25 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e4a8fa357a74064b1bafd7fc68b34d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f4c6c6fb9aa4d8bbccb659e2927fe55', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:47.248965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a074e8-1af6-42f0-9471-af46b2d24d25 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:47.248965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a074e8-1af6-42f0-9471-af46b2d24d25 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:47.248965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a074e8-1af6-42f0-9471-af46b2d24d25 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:47.248965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a074e8-1af6-42f0-9471-af46b2d24d25 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:47.248965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a074e8-1af6-42f0-9471-af46b2d24d25 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:47.249318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a074e8-1af6-42f0-9471-af46b2d24d25 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:47.259743 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1302/2614] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:11:47.262553 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16119bc7-4b81-4491-8e12-3b4458fbf75a service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:47.272157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16119bc7-4b81-4491-8e12-3b4458fbf75a service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:47.277126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b6e43-5014-4c66-8539-b8e611aeaf95 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e4a8fa357a74064b1bafd7fc68b34d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f4c6c6fb9aa4d8bbccb659e2927fe55', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:47.278468 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b6e43-5014-4c66-8539-b8e611aeaf95 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:47.278758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b6e43-5014-4c66-8539-b8e611aeaf95 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:47.279053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b6e43-5014-4c66-8539-b8e611aeaf95 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:47.279705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b6e43-5014-4c66-8539-b8e611aeaf95 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:47.281604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b6e43-5014-4c66-8539-b8e611aeaf95 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:47.381093 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1313/2615] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:47 2022] POST /identity/v3/projects => generated 356 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:47.382329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a074e8-1af6-42f0-9471-af46b2d24d25 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:47.393808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a074e8-1af6-42f0-9471-af46b2d24d25 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:47.419370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:47.443595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b6e43-5014-4c66-8539-b8e611aeaf95 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:47.445903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c65b6e43-5014-4c66-8539-b8e611aeaf95 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:47.467248 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb3d47-32da-421e-8699-d209956ce0b3 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e4a8fa357a74064b1bafd7fc68b34d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f4c6c6fb9aa4d8bbccb659e2927fe55', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:47.467895 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb3d47-32da-421e-8699-d209956ce0b3 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:47.468177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb3d47-32da-421e-8699-d209956ce0b3 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:47.468455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bb3d47-32da-421e-8699-d209956ce0b3 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:47.469112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb3d47-32da-421e-8699-d209956ce0b3 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:47.469417 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb3d47-32da-421e-8699-d209956ce0b3 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:47.471211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bb3d47-32da-421e-8699-d209956ce0b3 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:47.471981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:47.541284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b6e43-5014-4c66-8539-b8e611aeaf95 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:47.565294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:47.567335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:47.567335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:47.567335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] PATH_INFO: `/v3/users/1e6d1027d2724a04b56b9110f02cb03a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:47.568086 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6d1027d2724a04b56b9110f02cb03a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:47.570172 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:47.648403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1303/2616] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:11:47.693426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb3d47-32da-421e-8699-d209956ce0b3 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:47.705121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bb3d47-32da-421e-8699-d209956ce0b3 tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:47.710392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e4a8fa357a74064b1bafd7fc68b34d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5f4c6c6fb9aa4d8bbccb659e2927fe55', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:47.711136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:47.711419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:47.711686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] PATH_INFO: `/v3/projects/fb026ff3c8154c6a9e8ed51858dbe996` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:47.715859 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fb026ff3c8154c6a9e8ed51858dbe996)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:47.717618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:47.894846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6d1027d2724a04b56b9110f02cb03a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:47.895530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6d1027d2724a04b56b9110f02cb03a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:47.896230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6d1027d2724a04b56b9110f02cb03a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:47.901463 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb026ff3c8154c6a9e8ed51858dbe996', 'request_id': 'req-5e4413d5-5b03-4964-9a77-28192b22d89b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:47.908772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb026ff3c8154c6a9e8ed51858dbe996', 'request_id': 'req-5e4413d5-5b03-4964-9a77-28192b22d89b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:47.974755 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6d1027d2724a04b56b9110f02cb03a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:47.989539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6d1027d2724a04b56b9110f02cb03a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:48.002431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1314/2617] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:47 2022] DELETE /identity/v3/users/1e6d1027d2724a04b56b9110f02cb03a => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:48.098037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb026ff3c8154c6a9e8ed51858dbe996', 'request_id': 'req-5e4413d5-5b03-4964-9a77-28192b22d89b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:48.111723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] The token cache is being invalidate because project fb026ff3c8154c6a9e8ed51858dbe996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:48.112141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:48.113356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:48.118648 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1304/2618] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:47 2022] DELETE /identity/v3/projects/fb026ff3c8154c6a9e8ed51858dbe996 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:48.288020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:48.292047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:48.395835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fa248-685d-4581-a0a8-e3e14e7c9b05 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:48.405013 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:48.405684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:48.405960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:48.406225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] PATH_INFO: `/v3/projects/335b48b79b384dd5bfde02ebe0c18a14` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:48.408865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=335b48b79b384dd5bfde02ebe0c18a14)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:48.411101 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:48.609423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:48.619808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '335b48b79b384dd5bfde02ebe0c18a14', 'request_id': 'req-bc132a7c-8940-461c-ab53-2944df782351'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:48.627163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '335b48b79b384dd5bfde02ebe0c18a14', 'request_id': 'req-bc132a7c-8940-461c-ab53-2944df782351'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:48.633269 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '335b48b79b384dd5bfde02ebe0c18a14', 'request_id': 'req-bc132a7c-8940-461c-ab53-2944df782351'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:48.769713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] The token cache is being invalidate because project 335b48b79b384dd5bfde02ebe0c18a14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:48.770210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:48.770875 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:48.772874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1315/2619] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:48 2022] DELETE /identity/v3/projects/335b48b79b384dd5bfde02ebe0c18a14 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:48.777002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:48.779198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:48.853803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4413d5-5b03-4964-9a77-28192b22d89b tempest-VolumeQuotaClassesTest-1345854634 tempest-VolumeQuotaClassesTest-1345854634-project-admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:48.902224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:48.914534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:48.915191 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:48.915465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:48.915735 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] PATH_INFO: `/v3/users/02a7a72e1d3a407bbd09e10a05ef60c0` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:48.916864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a7a72e1d3a407bbd09e10a05ef60c0)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:48.918810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:49.054959 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:49.056373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:49.070096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc132a7c-8940-461c-ab53-2944df782351 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:49.076182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:49.076819 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:49.077100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:49.077373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] PATH_INFO: `/v3/users/d1d5192fc6024c4182ef9a29f675e6ca` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:49.078598 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d5192fc6024c4182ef9a29f675e6ca)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:49.080571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:49.212030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a7a72e1d3a407bbd09e10a05ef60c0'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.212685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a7a72e1d3a407bbd09e10a05ef60c0'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.213291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a7a72e1d3a407bbd09e10a05ef60c0'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.295647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d5192fc6024c4182ef9a29f675e6ca'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.296360 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d5192fc6024c4182ef9a29f675e6ca'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.296970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d5192fc6024c4182ef9a29f675e6ca'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.305940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a7a72e1d3a407bbd09e10a05ef60c0'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.321359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8cfdda25-dbe4-4532-ac8f-01dae9ab1efa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a7a72e1d3a407bbd09e10a05ef60c0'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.333856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1305/2620] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:48 2022] DELETE /identity/v3/users/02a7a72e1d3a407bbd09e10a05ef60c0 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:49.460448 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d5192fc6024c4182ef9a29f675e6ca'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.475080 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d5192fc6024c4182ef9a29f675e6ca'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.489750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1316/2621] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:49 2022] DELETE /identity/v3/users/d1d5192fc6024c4182ef9a29f675e6ca => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:49.493973 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85cee1-8a85-4e32-99a3-68d3be00a032 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:49.494256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85cee1-8a85-4e32-99a3-68d3be00a032 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:49.494524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85cee1-8a85-4e32-99a3-68d3be00a032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:49.517468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-7d85cee1-8a85-4e32-99a3-68d3be00a032 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:49.518846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7d85cee1-8a85-4e32-99a3-68d3be00a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:49.579589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1306/2622] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:49.589184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:49.590726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:49.616712 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85cee1-8a85-4e32-99a3-68d3be00a032 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:49.619157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7d85cee1-8a85-4e32-99a3-68d3be00a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:49.634290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85cee1-8a85-4e32-99a3-68d3be00a032 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:49.640841 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:49.641494 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:49.641771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:49.642062 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] PATH_INFO: `/v3/projects/35bcca433a86411ca5c1a55285805c93` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:49.643848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35bcca433a86411ca5c1a55285805c93)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:49.645770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:49.681852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac51d4-5b1d-4098-b336-26a554517cdd admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:49.685580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d25804-d51e-49a6-ac6f-b8c727267432 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:49.686259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25804-d51e-49a6-ac6f-b8c727267432 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:49.686559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25804-d51e-49a6-ac6f-b8c727267432 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:49.686835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d25804-d51e-49a6-ac6f-b8c727267432 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:49.696804 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d25804-d51e-49a6-ac6f-b8c727267432 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:49.698256 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d25804-d51e-49a6-ac6f-b8c727267432 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:49.708209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1317/2623] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:49.931302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35bcca433a86411ca5c1a55285805c93', 'request_id': 'req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:49.937853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35bcca433a86411ca5c1a55285805c93', 'request_id': 'req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:50.128966 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35bcca433a86411ca5c1a55285805c93', 'request_id': 'req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:50.131929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:50.141946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] The token cache is being invalidate because project 35bcca433a86411ca5c1a55285805c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:50.142237 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:50.142957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-03ec5e4b-e4de-49fc-b9ee-7f5d3e745d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:50.144495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1307/2624] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:49 2022] DELETE /identity/v3/projects/35bcca433a86411ca5c1a55285805c93 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:50.149971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878a809-011e-4f35-b7fc-a6324e52483d None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.150257 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878a809-011e-4f35-b7fc-a6324e52483d None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.150533 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878a809-011e-4f35-b7fc-a6324e52483d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.173624 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-0878a809-011e-4f35-b7fc-a6324e52483d None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:50.174926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-0878a809-011e-4f35-b7fc-a6324e52483d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:50.218019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:50.244760 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1318/2625] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:50.252983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897881d9-178a-4375-a2f1-679855da8106 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.253511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897881d9-178a-4375-a2f1-679855da8106 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.253962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897881d9-178a-4375-a2f1-679855da8106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.275338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-897881d9-178a-4375-a2f1-679855da8106 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:50.276543 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-897881d9-178a-4375-a2f1-679855da8106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:50.286257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1308/2626] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:50.290665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878a809-011e-4f35-b7fc-a6324e52483d None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:50.301191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878a809-011e-4f35-b7fc-a6324e52483d None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:50.307355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b529e99-187f-4e58-bd2b-12f5d00bd2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:50.307948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b529e99-187f-4e58-bd2b-12f5d00bd2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.308227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b529e99-187f-4e58-bd2b-12f5d00bd2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.308515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b529e99-187f-4e58-bd2b-12f5d00bd2a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.309208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b529e99-187f-4e58-bd2b-12f5d00bd2a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:50.309509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b529e99-187f-4e58-bd2b-12f5d00bd2a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:50.311202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b529e99-187f-4e58-bd2b-12f5d00bd2a3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:50.328781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1319/2627] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:11:50.332972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897881d9-178a-4375-a2f1-679855da8106 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:50.352835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897881d9-178a-4375-a2f1-679855da8106 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:50.358553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6d165-75cf-40c0-a3d7-38a5d0eb3c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:50.359201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6d165-75cf-40c0-a3d7-38a5d0eb3c90 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.359479 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6d165-75cf-40c0-a3d7-38a5d0eb3c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.359754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6d165-75cf-40c0-a3d7-38a5d0eb3c90 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.360392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6d165-75cf-40c0-a3d7-38a5d0eb3c90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:50.362266 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6d165-75cf-40c0-a3d7-38a5d0eb3c90 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:50.471430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1309/2628] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:50 2022] POST /identity/v3/projects => generated 386 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:50.475512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b529e99-187f-4e58-bd2b-12f5d00bd2a3 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:50.486311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b529e99-187f-4e58-bd2b-12f5d00bd2a3 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:50.545295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c6484-6ba1-49d6-9c6e-3385fb46fb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:50.546036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c6484-6ba1-49d6-9c6e-3385fb46fb25 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.546374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c6484-6ba1-49d6-9c6e-3385fb46fb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.546644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c6484-6ba1-49d6-9c6e-3385fb46fb25 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.547358 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c6484-6ba1-49d6-9c6e-3385fb46fb25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:50.549314 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c6484-6ba1-49d6-9c6e-3385fb46fb25 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:50.582471 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6d165-75cf-40c0-a3d7-38a5d0eb3c90 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:50.584473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2bf6d165-75cf-40c0-a3d7-38a5d0eb3c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:50.600385 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6d165-75cf-40c0-a3d7-38a5d0eb3c90 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:50.609958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:50.609958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.609958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.609958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] PATH_INFO: `/v3/projects/e68f9553fd4945f0bc7e8152a298b53e` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.610756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e68f9553fd4945f0bc7e8152a298b53e)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:50.612120 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:50.677904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1320/2629] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:50 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:50.679369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c6484-6ba1-49d6-9c6e-3385fb46fb25 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:50.690119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c6484-6ba1-49d6-9c6e-3385fb46fb25 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:50.695243 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439eff16-40b8-4ef7-a0dd-53931dbfc757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:50.695847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439eff16-40b8-4ef7-a0dd-53931dbfc757 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.696125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439eff16-40b8-4ef7-a0dd-53931dbfc757 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.696396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439eff16-40b8-4ef7-a0dd-53931dbfc757 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.697031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439eff16-40b8-4ef7-a0dd-53931dbfc757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:50.700517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439eff16-40b8-4ef7-a0dd-53931dbfc757 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:50.712593 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1321/2630] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:50 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:11:50.718610 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439eff16-40b8-4ef7-a0dd-53931dbfc757 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:50.728977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439eff16-40b8-4ef7-a0dd-53931dbfc757 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:50.735182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd95b1-69ca-496a-a671-916c23c8e894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:50.735816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd95b1-69ca-496a-a671-916c23c8e894 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.737828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd95b1-69ca-496a-a671-916c23c8e894 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.737828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd95b1-69ca-496a-a671-916c23c8e894 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.737828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd95b1-69ca-496a-a671-916c23c8e894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:50.738949 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd95b1-69ca-496a-a671-916c23c8e894 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:50.752999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1322/2631] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:50 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:11:50.754046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd95b1-69ca-496a-a671-916c23c8e894 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:50.767843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd95b1-69ca-496a-a671-916c23c8e894 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:50.784206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e68f9553fd4945f0bc7e8152a298b53e', 'request_id': 'req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:50.794114 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e68f9553fd4945f0bc7e8152a298b53e', 'request_id': 'req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:50.837264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a3c364-5bd3-4ac9-988f-09896069ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:50.837934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3c364-5bd3-4ac9-988f-09896069ddbd admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.838212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3c364-5bd3-4ac9-988f-09896069ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.838484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a3c364-5bd3-4ac9-988f-09896069ddbd admin admin] PATH_INFO: `/v3/projects/438f0597d07d49dcb4ba1d891114e06a/users/5e6340520c0041d28c4a208a08071984/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.853557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3c364-5bd3-4ac9-988f-09896069ddbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=438f0597d07d49dcb4ba1d891114e06a, user_id=5e6340520c0041d28c4a208a08071984, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:50.856245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a3c364-5bd3-4ac9-988f-09896069ddbd admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:50.900509 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e68f9553fd4945f0bc7e8152a298b53e', 'request_id': 'req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:50.912366 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] The token cache is being invalidate because project e68f9553fd4945f0bc7e8152a298b53e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:50.912654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:50.913297 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:50.914908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1310/2632] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:50 2022] DELETE /identity/v3/projects/e68f9553fd4945f0bc7e8152a298b53e => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:50.919050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:50.920276 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:50.933049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f7a90-4541-41aa-9e9c-5c0d956efaa3 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:50.938483 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:50.939149 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.939456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.939731 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] PATH_INFO: `/v3/users/6e4a8fa357a74064b1bafd7fc68b34d6` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.940946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4a8fa357a74064b1bafd7fc68b34d6)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:50.943109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:50.944913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1323/2633] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:50 2022] PUT /identity/v3/projects/438f0597d07d49dcb4ba1d891114e06a/users/5e6340520c0041d28c4a208a08071984/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:50.962421 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed19bbb-0ac0-47cb-86f6-05bf58e56f95 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:50.962709 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed19bbb-0ac0-47cb-86f6-05bf58e56f95 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:50.962982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed19bbb-0ac0-47cb-86f6-05bf58e56f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:50.990732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-8ed19bbb-0ac0-47cb-86f6-05bf58e56f95 None None] MFA Rules not processed for user `5e6340520c0041d28c4a208a08071984`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:50.991999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8ed19bbb-0ac0-47cb-86f6-05bf58e56f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:50.997067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:51.055927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1324/2634] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:50 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:11:51.067298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed19bbb-0ac0-47cb-86f6-05bf58e56f95 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:51.068683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8ed19bbb-0ac0-47cb-86f6-05bf58e56f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:51.129340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed19bbb-0ac0-47cb-86f6-05bf58e56f95 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:51.133712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ecc836-94fc-4615-91a0-793ede7f1f34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:51.134328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecc836-94fc-4615-91a0-793ede7f1f34 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:51.134877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecc836-94fc-4615-91a0-793ede7f1f34 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:51.135184 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ecc836-94fc-4615-91a0-793ede7f1f34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:51.144921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ecc836-94fc-4615-91a0-793ede7f1f34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:51.146375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ecc836-94fc-4615-91a0-793ede7f1f34 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:51.156961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1325/2635] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:51 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:11:51.178039 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ecc836-94fc-4615-91a0-793ede7f1f34 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:51.179618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f3ecc836-94fc-4615-91a0-793ede7f1f34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:51.247595 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ecc836-94fc-4615-91a0-793ede7f1f34 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:51.248972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4a8fa357a74064b1bafd7fc68b34d6'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:51.249635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4a8fa357a74064b1bafd7fc68b34d6'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:51.250340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4a8fa357a74064b1bafd7fc68b34d6'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:51.252305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:51.253054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:51.253482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:51.253761 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] PATH_INFO: `/v3/users/5e6340520c0041d28c4a208a08071984` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:51.254914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e6340520c0041d28c4a208a08071984)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:51.257014 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:51.328625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4a8fa357a74064b1bafd7fc68b34d6'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:51.342567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f7652457-21cf-4c07-969e-98c2a1441dbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4a8fa357a74064b1bafd7fc68b34d6'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:51.355810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1311/2636] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:50 2022] DELETE /identity/v3/users/6e4a8fa357a74064b1bafd7fc68b34d6 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:51.567359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e6340520c0041d28c4a208a08071984'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:51.568020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e6340520c0041d28c4a208a08071984'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:51.568627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e6340520c0041d28c4a208a08071984'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:51.572824 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:51.734914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e6340520c0041d28c4a208a08071984'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:51.748616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e6340520c0041d28c4a208a08071984'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:51.760632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1326/2637] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:51 2022] DELETE /identity/v3/users/5e6340520c0041d28c4a208a08071984 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:51.766423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a6a96-3dec-4dd2-8708-9f579b99fad9 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:51.766423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a6a96-3dec-4dd2-8708-9f579b99fad9 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:51.766423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a6a96-3dec-4dd2-8708-9f579b99fad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:51.787396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-a19a6a96-3dec-4dd2-8708-9f579b99fad9 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:51.788619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a19a6a96-3dec-4dd2-8708-9f579b99fad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:51.801090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=79,keystone:DELETE=18,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:51.857253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1312/2638] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:51.873392 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:51.875750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:51.881850 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a6a96-3dec-4dd2-8708-9f579b99fad9 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:51.884418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a19a6a96-3dec-4dd2-8708-9f579b99fad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:51.892854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559b3fe-6ebe-4f88-8060-185ef943b7bf admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:51.899614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:51.899614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:51.899614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:51.899614 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] PATH_INFO: `/v3/projects/5f4c6c6fb9aa4d8bbccb659e2927fe55` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:51.900749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f4c6c6fb9aa4d8bbccb659e2927fe55)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:51.903275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:51.974477 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a6a96-3dec-4dd2-8708-9f579b99fad9 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:51.978486 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805e0d3-75b3-4e26-b668-f964be8b985f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:51.979136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805e0d3-75b3-4e26-b668-f964be8b985f service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:51.979544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805e0d3-75b3-4e26-b668-f964be8b985f service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:51.979698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805e0d3-75b3-4e26-b668-f964be8b985f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:51.991045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805e0d3-75b3-4e26-b668-f964be8b985f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:51.992391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805e0d3-75b3-4e26-b668-f964be8b985f service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:52.003928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1313/2639] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:11:52.049571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f4c6c6fb9aa4d8bbccb659e2927fe55', 'request_id': 'req-31e09be5-6397-4499-92cb-84fa61bec5fc'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:52.056859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f4c6c6fb9aa4d8bbccb659e2927fe55', 'request_id': 'req-31e09be5-6397-4499-92cb-84fa61bec5fc'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:52.065988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f4c6c6fb9aa4d8bbccb659e2927fe55', 'request_id': 'req-31e09be5-6397-4499-92cb-84fa61bec5fc'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:52.165581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] The token cache is being invalidate because project 5f4c6c6fb9aa4d8bbccb659e2927fe55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:52.165854 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:52.166508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-31e09be5-6397-4499-92cb-84fa61bec5fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:52.168434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1327/2640] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:51 2022] DELETE /identity/v3/projects/5f4c6c6fb9aa4d8bbccb659e2927fe55 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:52.174007 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed59e0c-844f-4d90-b565-46cd07166c9a None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.174348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed59e0c-844f-4d90-b565-46cd07166c9a None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.174619 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed59e0c-844f-4d90-b565-46cd07166c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.197150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-bed59e0c-844f-4d90-b565-46cd07166c9a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:52.198311 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-bed59e0c-844f-4d90-b565-46cd07166c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:52.282911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1314/2641] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:52.291125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e6a46-19aa-447c-9b54-f7528ec6246b None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.294812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e6a46-19aa-447c-9b54-f7528ec6246b None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.297811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e6a46-19aa-447c-9b54-f7528ec6246b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.323079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f95e6a46-19aa-447c-9b54-f7528ec6246b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:52.324777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f95e6a46-19aa-447c-9b54-f7528ec6246b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:52.337024 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1328/2642] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:52.338864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed59e0c-844f-4d90-b565-46cd07166c9a None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:52.357261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed59e0c-844f-4d90-b565-46cd07166c9a None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:52.365695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e0a05-02eb-40d1-a9db-703124351b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:52.366474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0a05-02eb-40d1-a9db-703124351b0c admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.366808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0a05-02eb-40d1-a9db-703124351b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.367187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0a05-02eb-40d1-a9db-703124351b0c admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.367984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e0a05-02eb-40d1-a9db-703124351b0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:52.368248 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e0a05-02eb-40d1-a9db-703124351b0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:52.370052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e0a05-02eb-40d1-a9db-703124351b0c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:52.384964 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1315/2643] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:11:52.390813 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e6a46-19aa-447c-9b54-f7528ec6246b None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:52.402462 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e6a46-19aa-447c-9b54-f7528ec6246b None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:52.407586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5867f2-7256-406a-90b9-206e4fa7789d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:52.408221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5867f2-7256-406a-90b9-206e4fa7789d admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.408487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5867f2-7256-406a-90b9-206e4fa7789d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.408765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5867f2-7256-406a-90b9-206e4fa7789d admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.409427 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5867f2-7256-406a-90b9-206e4fa7789d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:52.411575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5867f2-7256-406a-90b9-206e4fa7789d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:52.549484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1329/2644] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:52 2022] POST /identity/v3/projects => generated 368 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:52.555706 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5867f2-7256-406a-90b9-206e4fa7789d admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:52.555923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e0a05-02eb-40d1-a9db-703124351b0c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:52.559454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-c22e0a05-02eb-40d1-a9db-703124351b0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:52.594501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e0a05-02eb-40d1-a9db-703124351b0c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:52.650359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:52.658444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5867f2-7256-406a-90b9-206e4fa7789d admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:52.663944 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e2b3b-438c-4783-a98b-00bb38723d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:52.664569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e2b3b-438c-4783-a98b-00bb38723d69 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.664865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e2b3b-438c-4783-a98b-00bb38723d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.665259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e2b3b-438c-4783-a98b-00bb38723d69 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.665756 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e2b3b-438c-4783-a98b-00bb38723d69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:52.670349 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:52.670577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182e2b3b-438c-4783-a98b-00bb38723d69 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:52.670692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.670692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.670692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] PATH_INFO: `/v3/projects/438f0597d07d49dcb4ba1d891114e06a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.672123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=438f0597d07d49dcb4ba1d891114e06a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:52.675396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:52.785812 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1330/2645] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:52 2022] POST /identity/v3/users => generated 285 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:52.787384 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e2b3b-438c-4783-a98b-00bb38723d69 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:52.796482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '438f0597d07d49dcb4ba1d891114e06a', 'request_id': 'req-209d8b53-55cb-44cb-a540-f7b333178533'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:52.799301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e2b3b-438c-4783-a98b-00bb38723d69 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:52.803613 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '438f0597d07d49dcb4ba1d891114e06a', 'request_id': 'req-209d8b53-55cb-44cb-a540-f7b333178533'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:52.809412 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:52.862721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1d770-db0f-4f66-8f8e-8142ddc461a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:52.863404 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1d770-db0f-4f66-8f8e-8142ddc461a7 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.863683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1d770-db0f-4f66-8f8e-8142ddc461a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.863948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1d770-db0f-4f66-8f8e-8142ddc461a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.864615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1d770-db0f-4f66-8f8e-8142ddc461a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:52.866197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc1d770-db0f-4f66-8f8e-8142ddc461a7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:52.877255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1331/2646] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:52 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:11:52.881654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1d770-db0f-4f66-8f8e-8142ddc461a7 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:52.892574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1d770-db0f-4f66-8f8e-8142ddc461a7 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:52.897556 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825812d-d251-4f3f-903a-e27d1d681977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:52.898178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825812d-d251-4f3f-903a-e27d1d681977 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.898459 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825812d-d251-4f3f-903a-e27d1d681977 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.898727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825812d-d251-4f3f-903a-e27d1d681977 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.899403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825812d-d251-4f3f-903a-e27d1d681977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:52.900999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825812d-d251-4f3f-903a-e27d1d681977 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:52.903109 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '438f0597d07d49dcb4ba1d891114e06a', 'request_id': 'req-209d8b53-55cb-44cb-a540-f7b333178533'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:52.915447 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1332/2647] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:52 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:11:52.920204 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825812d-d251-4f3f-903a-e27d1d681977 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:52.923058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] The token cache is being invalidate because project 438f0597d07d49dcb4ba1d891114e06a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:52.923614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:52.924521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-209d8b53-55cb-44cb-a540-f7b333178533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:52.926212 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1316/2648] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:52 2022] DELETE /identity/v3/projects/438f0597d07d49dcb4ba1d891114e06a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:52.931242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf048-689e-40cb-9f01-6cbd348376ac None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.931546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825812d-d251-4f3f-903a-e27d1d681977 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:52.931947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf048-689e-40cb-9f01-6cbd348376ac None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.932408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf048-689e-40cb-9f01-6cbd348376ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.937104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5fbbb-07cc-4972-81e6-12246faa96d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:52.937899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5fbbb-07cc-4972-81e6-12246faa96d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.938189 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5fbbb-07cc-4972-81e6-12246faa96d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.938467 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5fbbb-07cc-4972-81e6-12246faa96d5 admin admin] PATH_INFO: `/v3/projects/d4ef92db2d3b4157913b1d428ecdbb1e/users/b08f4e3a7d84483991f0a77dcff02af0/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.955127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5fbbb-07cc-4972-81e6-12246faa96d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4ef92db2d3b4157913b1d428ecdbb1e, user_id=b08f4e3a7d84483991f0a77dcff02af0, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:52.958050 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5fbbb-07cc-4972-81e6-12246faa96d5 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:52.958201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-496bf048-689e-40cb-9f01-6cbd348376ac None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:52.959382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-496bf048-689e-40cb-9f01-6cbd348376ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:52.968068 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1317/2649] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:52.975654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007386d3-98ad-4944-a76e-3561f592979c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:52.976254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007386d3-98ad-4944-a76e-3561f592979c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:52.976254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007386d3-98ad-4944-a76e-3561f592979c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:52.997223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-007386d3-98ad-4944-a76e-3561f592979c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:52.998557 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-007386d3-98ad-4944-a76e-3561f592979c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:53.006580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1318/2650] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:11:53.010455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007386d3-98ad-4944-a76e-3561f592979c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:53.019784 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007386d3-98ad-4944-a76e-3561f592979c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:53.024281 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395b989-4bdb-4506-a040-6261c1abea02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:53.024865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395b989-4bdb-4506-a040-6261c1abea02 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.025139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395b989-4bdb-4506-a040-6261c1abea02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.025406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395b989-4bdb-4506-a040-6261c1abea02 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.026039 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395b989-4bdb-4506-a040-6261c1abea02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:53.026333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395b989-4bdb-4506-a040-6261c1abea02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:53.028435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395b989-4bdb-4506-a040-6261c1abea02 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:53.040003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1319/2651] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:11:53.044803 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395b989-4bdb-4506-a040-6261c1abea02 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:53.054386 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395b989-4bdb-4506-a040-6261c1abea02 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:53.058927 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf85e53-af9e-462a-af61-3b7f446cbb64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:53.059524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf85e53-af9e-462a-af61-3b7f446cbb64 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.059797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf85e53-af9e-462a-af61-3b7f446cbb64 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.060060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf85e53-af9e-462a-af61-3b7f446cbb64 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.060655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf85e53-af9e-462a-af61-3b7f446cbb64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:53.062589 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf85e53-af9e-462a-af61-3b7f446cbb64 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:53.072234 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1333/2652] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:52 2022] PUT /identity/v3/projects/d4ef92db2d3b4157913b1d428ecdbb1e/users/b08f4e3a7d84483991f0a77dcff02af0/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:11:53.085612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a676f-9e35-454b-aa8a-36059e6e1360 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.085893 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a676f-9e35-454b-aa8a-36059e6e1360 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.086165 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a676f-9e35-454b-aa8a-36059e6e1360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.114198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-727a676f-9e35-454b-aa8a-36059e6e1360 None None] MFA Rules not processed for user `b08f4e3a7d84483991f0a77dcff02af0`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:53.115439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-727a676f-9e35-454b-aa8a-36059e6e1360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:53.181142 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1334/2653] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:11:53.184296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1320/2654] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:53 2022] POST /identity/v3/projects => generated 360 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:11:53.188788 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a676f-9e35-454b-aa8a-36059e6e1360 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:53.195548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf85e53-af9e-462a-af61-3b7f446cbb64 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:53.197234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-bbf85e53-af9e-462a-af61-3b7f446cbb64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:53.257756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf85e53-af9e-462a-af61-3b7f446cbb64 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:53.260340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a676f-9e35-454b-aa8a-36059e6e1360 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:53.261449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617eac4a-4b70-469e-bb0c-18a06c9f6961 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d62a4c07c4402ea5803f6e68fb1d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:53.262844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617eac4a-4b70-469e-bb0c-18a06c9f6961 service glance] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.263318 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617eac4a-4b70-469e-bb0c-18a06c9f6961 service glance] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.263757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617eac4a-4b70-469e-bb0c-18a06c9f6961 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.269486 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:53.311416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:53.321311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9b2cf-7356-440c-a8a1-fa3da8530d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:53.321951 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9b2cf-7356-440c-a8a1-fa3da8530d6f admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.322259 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9b2cf-7356-440c-a8a1-fa3da8530d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.322536 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9b2cf-7356-440c-a8a1-fa3da8530d6f admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.323207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9b2cf-7356-440c-a8a1-fa3da8530d6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:53.325312 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617eac4a-4b70-469e-bb0c-18a06c9f6961 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:53.325710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9b2cf-7356-440c-a8a1-fa3da8530d6f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:53.326871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617eac4a-4b70-469e-bb0c-18a06c9f6961 service glance] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:53.337389 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1321/2655] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:11:53.453567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1335/2656] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:53 2022] POST /identity/v3/users => generated 281 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:11:53.457660 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617eac4a-4b70-469e-bb0c-18a06c9f6961 service glance] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:53.468684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617eac4a-4b70-469e-bb0c-18a06c9f6961 service glance] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:53.474688 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7f1e8-3581-4961-a7b7-6351ee99c321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:53.475323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7f1e8-3581-4961-a7b7-6351ee99c321 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.475601 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7f1e8-3581-4961-a7b7-6351ee99c321 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.475866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7f1e8-3581-4961-a7b7-6351ee99c321 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.476537 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7f1e8-3581-4961-a7b7-6351ee99c321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:53.478112 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7f1e8-3581-4961-a7b7-6351ee99c321 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:53.488894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1322/2657] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:53 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:53.493054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9b2cf-7356-440c-a8a1-fa3da8530d6f admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:53.504138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9b2cf-7356-440c-a8a1-fa3da8530d6f admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:53.509813 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667dc173-54f5-4432-8a9d-045e59e6a175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:53.509813 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dc173-54f5-4432-8a9d-045e59e6a175 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.510405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dc173-54f5-4432-8a9d-045e59e6a175 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.510405 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667dc173-54f5-4432-8a9d-045e59e6a175 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.511695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dc173-54f5-4432-8a9d-045e59e6a175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:53.513512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667dc173-54f5-4432-8a9d-045e59e6a175 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:53.527399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1336/2658] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:53 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:11:53.528804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7f1e8-3581-4961-a7b7-6351ee99c321 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:53.542488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7f1e8-3581-4961-a7b7-6351ee99c321 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:53.548359 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd552ce-2326-42d9-8e06-84da754ec8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:53.549042 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd552ce-2326-42d9-8e06-84da754ec8f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.549348 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd552ce-2326-42d9-8e06-84da754ec8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.549594 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd552ce-2326-42d9-8e06-84da754ec8f9 admin admin] PATH_INFO: `/v3/projects/75cce4a547bc4091a0db9738bb2cd76c/users/8494f30bfeb7430784b72d74f0d6e35d/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.567427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd552ce-2326-42d9-8e06-84da754ec8f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75cce4a547bc4091a0db9738bb2cd76c, user_id=8494f30bfeb7430784b72d74f0d6e35d, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:53.570066 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd552ce-2326-42d9-8e06-84da754ec8f9 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:53.664336 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1323/2659] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:53 2022] PUT /identity/v3/projects/75cce4a547bc4091a0db9738bb2cd76c/users/8494f30bfeb7430784b72d74f0d6e35d/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:53.675000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178f41e-0605-405c-bf55-6830411b8108 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.675513 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178f41e-0605-405c-bf55-6830411b8108 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.675750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178f41e-0605-405c-bf55-6830411b8108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.705577 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd552ce-2326-42d9-8e06-84da754ec8f9 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:53.707421 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7dd552ce-2326-42d9-8e06-84da754ec8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:53.714178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-3178f41e-0605-405c-bf55-6830411b8108 None None] MFA Rules not processed for user `8494f30bfeb7430784b72d74f0d6e35d`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:53.715564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3178f41e-0605-405c-bf55-6830411b8108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:53.781488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd552ce-2326-42d9-8e06-84da754ec8f9 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:53.785947 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08492fe4-3754-4559-9e81-c766fbe48390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:53.786697 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08492fe4-3754-4559-9e81-c766fbe48390 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.787003 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08492fe4-3754-4559-9e81-c766fbe48390 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.792507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08492fe4-3754-4559-9e81-c766fbe48390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.792507 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-08492fe4-3754-4559-9e81-c766fbe48390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:53.810631 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1337/2660] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:53.858017 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178f41e-0605-405c-bf55-6830411b8108 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:53.860273 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3178f41e-0605-405c-bf55-6830411b8108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:53.885590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08492fe4-3754-4559-9e81-c766fbe48390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:53.896799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08492fe4-3754-4559-9e81-c766fbe48390 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:53.901936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1324/2661] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:11:53.935501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178f41e-0605-405c-bf55-6830411b8108 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:53.940006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c094e-8f94-4c25-b120-722e2fba105d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:53.940732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c094e-8f94-4c25-b120-722e2fba105d service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:53.940999 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c094e-8f94-4c25-b120-722e2fba105d service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:53.941514 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c094e-8f94-4c25-b120-722e2fba105d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:53.955721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c094e-8f94-4c25-b120-722e2fba105d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:53.957196 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c094e-8f94-4c25-b120-722e2fba105d service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:53.968509 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1338/2662] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:53.986234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08492fe4-3754-4559-9e81-c766fbe48390 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:54.000332 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08492fe4-3754-4559-9e81-c766fbe48390 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:54.006056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:54.007083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:54.007306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:54.007565 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] PATH_INFO: `/v3/users/8494f30bfeb7430784b72d74f0d6e35d` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:54.009568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] RBAC: Authorizing `identity:delete_user(user_id=8494f30bfeb7430784b72d74f0d6e35d)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:54.013139 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:54.270201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8494f30bfeb7430784b72d74f0d6e35d'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:54.270934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8494f30bfeb7430784b72d74f0d6e35d'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:54.272354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8494f30bfeb7430784b72d74f0d6e35d'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:54.392296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8494f30bfeb7430784b72d74f0d6e35d'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:54.394807 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:54.415662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8494f30bfeb7430784b72d74f0d6e35d'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:54.429226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1325/2663] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:53 2022] DELETE /identity/v3/users/8494f30bfeb7430784b72d74f0d6e35d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:54.434692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e61a-b503-4afa-a418-8925416ce562 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:54.434968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e61a-b503-4afa-a418-8925416ce562 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:54.435262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f144e61a-b503-4afa-a418-8925416ce562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:54.458337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f144e61a-b503-4afa-a418-8925416ce562 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:54.459615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f144e61a-b503-4afa-a418-8925416ce562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:54.482216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:54.537345 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1339/2664] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:54.555777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:54.555777 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:54.626582 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bbed59-6dfc-468d-830d-de17855b79de admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:54.630449 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86818d1b-857c-4db2-9b6a-44c1c9f35a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:54.631097 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86818d1b-857c-4db2-9b6a-44c1c9f35a36 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:54.631533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86818d1b-857c-4db2-9b6a-44c1c9f35a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:54.631813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86818d1b-857c-4db2-9b6a-44c1c9f35a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:54.643434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86818d1b-857c-4db2-9b6a-44c1c9f35a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:54.645100 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86818d1b-857c-4db2-9b6a-44c1c9f35a36 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:54.655984 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1326/2665] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:11:55.263527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144e61a-b503-4afa-a418-8925416ce562 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:55.269106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f144e61a-b503-4afa-a418-8925416ce562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:55.284530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f144e61a-b503-4afa-a418-8925416ce562 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:55.296297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:55.297619 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:55.297964 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:55.298461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] PATH_INFO: `/v3/projects/75cce4a547bc4091a0db9738bb2cd76c` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:55.299824 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75cce4a547bc4091a0db9738bb2cd76c)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:55.302127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:55.487742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75cce4a547bc4091a0db9738bb2cd76c', 'request_id': 'req-9683f457-15da-4b87-b73d-494f287a63f7'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:55.496199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75cce4a547bc4091a0db9738bb2cd76c', 'request_id': 'req-9683f457-15da-4b87-b73d-494f287a63f7'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:55.502347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75cce4a547bc4091a0db9738bb2cd76c', 'request_id': 'req-9683f457-15da-4b87-b73d-494f287a63f7'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:55.636001 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] The token cache is being invalidate because project 75cce4a547bc4091a0db9738bb2cd76c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:55.636454 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:55.637204 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9683f457-15da-4b87-b73d-494f287a63f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:55.640816 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1340/2666] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:55 2022] DELETE /identity/v3/projects/75cce4a547bc4091a0db9738bb2cd76c => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:55.645517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111992c5-971f-42ba-bee5-d9edc7f80720 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:55.646070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111992c5-971f-42ba-bee5-d9edc7f80720 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:55.646070 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111992c5-971f-42ba-bee5-d9edc7f80720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:55.671230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-111992c5-971f-42ba-bee5-d9edc7f80720 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:55.679145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-111992c5-971f-42ba-bee5-d9edc7f80720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:55.768012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1327/2667] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:55.776814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a04f5-7e72-4c1f-bb9a-2f1e0d1caf5f None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:55.777100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a04f5-7e72-4c1f-bb9a-2f1e0d1caf5f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:55.777373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a04f5-7e72-4c1f-bb9a-2f1e0d1caf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:55.800565 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-660a04f5-7e72-4c1f-bb9a-2f1e0d1caf5f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:55.801511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-660a04f5-7e72-4c1f-bb9a-2f1e0d1caf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:55.811612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1341/2668] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:55 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:55.816467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111992c5-971f-42ba-bee5-d9edc7f80720 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:55.829807 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111992c5-971f-42ba-bee5-d9edc7f80720 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:55.835445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea346cf9-4258-42c9-b7a5-1a815dec12be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:55.836516 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea346cf9-4258-42c9-b7a5-1a815dec12be admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:55.837018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea346cf9-4258-42c9-b7a5-1a815dec12be admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:55.837741 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea346cf9-4258-42c9-b7a5-1a815dec12be admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:55.838652 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea346cf9-4258-42c9-b7a5-1a815dec12be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:55.839493 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea346cf9-4258-42c9-b7a5-1a815dec12be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:55.843260 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea346cf9-4258-42c9-b7a5-1a815dec12be admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:55.864398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1328/2669] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:11:55.865808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a04f5-7e72-4c1f-bb9a-2f1e0d1caf5f None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:55.878428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a04f5-7e72-4c1f-bb9a-2f1e0d1caf5f None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:55.884865 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb23b-ce8b-4bf6-8e11-65f0c87533f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:55.885735 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb23b-ce8b-4bf6-8e11-65f0c87533f4 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:55.886203 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb23b-ce8b-4bf6-8e11-65f0c87533f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:55.886813 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb23b-ce8b-4bf6-8e11-65f0c87533f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:55.887795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcb23b-ce8b-4bf6-8e11-65f0c87533f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:55.890035 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcb23b-ce8b-4bf6-8e11-65f0c87533f4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:56.025979 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1342/2670] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:55 2022] POST /identity/v3/projects => generated 378 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:56.027074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea346cf9-4258-42c9-b7a5-1a815dec12be admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:56.039382 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea346cf9-4258-42c9-b7a5-1a815dec12be admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:56.111940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9f1e0-b9b2-4de7-ac53-6c186351e802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:56.112812 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9f1e0-b9b2-4de7-ac53-6c186351e802 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:56.113199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9f1e0-b9b2-4de7-ac53-6c186351e802 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:56.113580 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9f1e0-b9b2-4de7-ac53-6c186351e802 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:56.114326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9f1e0-b9b2-4de7-ac53-6c186351e802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:56.116423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9f1e0-b9b2-4de7-ac53-6c186351e802 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:56.126885 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:56.229671 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1329/2671] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:56 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:56.230904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb23b-ce8b-4bf6-8e11-65f0c87533f4 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:56.243780 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb23b-ce8b-4bf6-8e11-65f0c87533f4 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:56.249566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17da339-9789-456a-8a9d-7013d2a3ac15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:56.250331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17da339-9789-456a-8a9d-7013d2a3ac15 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:56.250676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17da339-9789-456a-8a9d-7013d2a3ac15 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:56.251069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17da339-9789-456a-8a9d-7013d2a3ac15 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:56.251793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17da339-9789-456a-8a9d-7013d2a3ac15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:56.253499 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17da339-9789-456a-8a9d-7013d2a3ac15 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:56.266518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1343/2672] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:56 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:56.288333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9f1e0-b9b2-4de7-ac53-6c186351e802 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:56.299505 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9f1e0-b9b2-4de7-ac53-6c186351e802 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:56.305576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db2b75-a159-4dc1-972c-15001a9d35f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:56.306360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db2b75-a159-4dc1-972c-15001a9d35f7 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:56.306738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db2b75-a159-4dc1-972c-15001a9d35f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:56.307229 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db2b75-a159-4dc1-972c-15001a9d35f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:56.307865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db2b75-a159-4dc1-972c-15001a9d35f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:56.309814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db2b75-a159-4dc1-972c-15001a9d35f7 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:56.322289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1330/2673] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:56 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:56.329457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17da339-9789-456a-8a9d-7013d2a3ac15 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:56.343316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17da339-9789-456a-8a9d-7013d2a3ac15 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:56.348759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a217675-cd83-40d4-9ff5-276fae1cd8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:56.349431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a217675-cd83-40d4-9ff5-276fae1cd8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:56.349710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a217675-cd83-40d4-9ff5-276fae1cd8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:56.349987 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a217675-cd83-40d4-9ff5-276fae1cd8f1 admin admin] PATH_INFO: `/v3/projects/58d483bf70a94a4b99b987c159e865df/users/346b16eccc4e4535b3482e22e115249d/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:56.374695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a217675-cd83-40d4-9ff5-276fae1cd8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d483bf70a94a4b99b987c159e865df, user_id=346b16eccc4e4535b3482e22e115249d, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:56.374695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a217675-cd83-40d4-9ff5-276fae1cd8f1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:56.461752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1344/2674] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:56 2022] PUT /identity/v3/projects/58d483bf70a94a4b99b987c159e865df/users/346b16eccc4e4535b3482e22e115249d/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:56.472532 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b65cb8-3420-4d7f-8799-8deb74d99e50 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:56.472829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b65cb8-3420-4d7f-8799-8deb74d99e50 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:56.473092 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b65cb8-3420-4d7f-8799-8deb74d99e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:56.503829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b6b65cb8-3420-4d7f-8799-8deb74d99e50 None None] MFA Rules not processed for user `346b16eccc4e4535b3482e22e115249d`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:56.505058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b6b65cb8-3420-4d7f-8799-8deb74d99e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:56.586740 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1331/2675] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:56.591632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a217675-cd83-40d4-9ff5-276fae1cd8f1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:56.639934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:56.656376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a217675-cd83-40d4-9ff5-276fae1cd8f1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:56.661753 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d49e4-c9fc-4ce8-aea5-02ce980db08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:56.662363 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d49e4-c9fc-4ce8-aea5-02ce980db08b admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:56.662646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d49e4-c9fc-4ce8-aea5-02ce980db08b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:56.662919 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d49e4-c9fc-4ce8-aea5-02ce980db08b admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:56.663580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d49e4-c9fc-4ce8-aea5-02ce980db08b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:56.665692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d49e4-c9fc-4ce8-aea5-02ce980db08b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:56.842211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1345/2676] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:56 2022] POST /identity/v3/projects => generated 376 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:56.844272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b65cb8-3420-4d7f-8799-8deb74d99e50 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:56.855371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b65cb8-3420-4d7f-8799-8deb74d99e50 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:56.926245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba42f3d-df63-4dad-bd2e-6ddf7d486d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:56.926866 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba42f3d-df63-4dad-bd2e-6ddf7d486d8b admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:56.927176 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba42f3d-df63-4dad-bd2e-6ddf7d486d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:56.927450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba42f3d-df63-4dad-bd2e-6ddf7d486d8b admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:56.928333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba42f3d-df63-4dad-bd2e-6ddf7d486d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:56.930327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba42f3d-df63-4dad-bd2e-6ddf7d486d8b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:57.025887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1332/2677] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:56 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:57.030036 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d49e4-c9fc-4ce8-aea5-02ce980db08b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:57.042784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d49e4-c9fc-4ce8-aea5-02ce980db08b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:57.049794 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab79d0-a668-4803-8c23-e6ae26bd58cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:57.050944 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab79d0-a668-4803-8c23-e6ae26bd58cd admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:57.051550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab79d0-a668-4803-8c23-e6ae26bd58cd admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:57.051989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab79d0-a668-4803-8c23-e6ae26bd58cd admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:57.053317 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab79d0-a668-4803-8c23-e6ae26bd58cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:57.055063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab79d0-a668-4803-8c23-e6ae26bd58cd admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:57.068531 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1346/2678] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:57 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:57.073285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba42f3d-df63-4dad-bd2e-6ddf7d486d8b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:57.090745 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba42f3d-df63-4dad-bd2e-6ddf7d486d8b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:57.097242 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44461555-fe8b-4bdf-9e24-e9d4bd938f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:57.097962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44461555-fe8b-4bdf-9e24-e9d4bd938f08 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:57.098518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44461555-fe8b-4bdf-9e24-e9d4bd938f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:57.099306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44461555-fe8b-4bdf-9e24-e9d4bd938f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:57.105522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44461555-fe8b-4bdf-9e24-e9d4bd938f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:57.105522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44461555-fe8b-4bdf-9e24-e9d4bd938f08 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:57.114467 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1333/2679] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:57 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:11:57.119354 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab79d0-a668-4803-8c23-e6ae26bd58cd admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:57.130646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab79d0-a668-4803-8c23-e6ae26bd58cd admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:57.135495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbdb9a-5fdd-4f3a-957a-4c9013e51531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:57.136379 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbdb9a-5fdd-4f3a-957a-4c9013e51531 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:57.137910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbdb9a-5fdd-4f3a-957a-4c9013e51531 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:57.137910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbdb9a-5fdd-4f3a-957a-4c9013e51531 admin admin] PATH_INFO: `/v3/projects/b1d620fff8b340ad9f79d511fbbb5336/users/08cc0465a9d34292b048b0ba31ccb690/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:57.161643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbdb9a-5fdd-4f3a-957a-4c9013e51531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d620fff8b340ad9f79d511fbbb5336, user_id=08cc0465a9d34292b048b0ba31ccb690, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:57.164378 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbdb9a-5fdd-4f3a-957a-4c9013e51531 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:57.334154 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1347/2680] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:57 2022] PUT /identity/v3/projects/b1d620fff8b340ad9f79d511fbbb5336/users/08cc0465a9d34292b048b0ba31ccb690/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:57.345180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68fed6-1a9f-4c9f-9732-ce5fe77c9a91 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:57.345548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68fed6-1a9f-4c9f-9732-ce5fe77c9a91 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:57.345828 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68fed6-1a9f-4c9f-9732-ce5fe77c9a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:57.384965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-8b68fed6-1a9f-4c9f-9732-ce5fe77c9a91 None None] MFA Rules not processed for user `08cc0465a9d34292b048b0ba31ccb690`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:57.386342 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8b68fed6-1a9f-4c9f-9732-ce5fe77c9a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:57.453211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:57.506914 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1334/2681] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:57.507763 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbdb9a-5fdd-4f3a-957a-4c9013e51531 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:57.603740 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbdb9a-5fdd-4f3a-957a-4c9013e51531 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:57.610019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f6336-3272-49de-9753-b6612aaed57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:57.610634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f6336-3272-49de-9753-b6612aaed57b admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:57.610911 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f6336-3272-49de-9753-b6612aaed57b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:57.611312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f6336-3272-49de-9753-b6612aaed57b admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:57.611961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f6336-3272-49de-9753-b6612aaed57b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:57.614213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6f6336-3272-49de-9753-b6612aaed57b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:57.771444 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1348/2682] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:57 2022] POST /identity/v3/projects => generated 378 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:11:57.780357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68fed6-1a9f-4c9f-9732-ce5fe77c9a91 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:57.796814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68fed6-1a9f-4c9f-9732-ce5fe77c9a91 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:57.866027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a1dc1-9154-4695-9041-d273de448c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:57.866027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a1dc1-9154-4695-9041-d273de448c93 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:57.866027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a1dc1-9154-4695-9041-d273de448c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:57.866027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a1dc1-9154-4695-9041-d273de448c93 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:57.866932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a1dc1-9154-4695-9041-d273de448c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:57.866932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a1dc1-9154-4695-9041-d273de448c93 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:58.033435 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1335/2683] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:57 2022] POST /identity/v3/users => generated 296 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:58.038125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f6336-3272-49de-9753-b6612aaed57b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:58.055094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f6336-3272-49de-9753-b6612aaed57b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:58.059493 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0e826-9315-4a20-9fcc-1b6815d3fbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:58.060164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0e826-9315-4a20-9fcc-1b6815d3fbd1 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:58.060512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0e826-9315-4a20-9fcc-1b6815d3fbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:58.060790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0e826-9315-4a20-9fcc-1b6815d3fbd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:58.061439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0e826-9315-4a20-9fcc-1b6815d3fbd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:58.063075 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0e826-9315-4a20-9fcc-1b6815d3fbd1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:58.078098 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1349/2684] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:58 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:58.084625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a1dc1-9154-4695-9041-d273de448c93 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:58.097422 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a1dc1-9154-4695-9041-d273de448c93 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:58.102286 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76905b-415d-4263-814f-1a53219d15b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:58.103053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76905b-415d-4263-814f-1a53219d15b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:58.103469 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76905b-415d-4263-814f-1a53219d15b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:58.103817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76905b-415d-4263-814f-1a53219d15b8 admin admin] PATH_INFO: `/v3/projects/f4261c49b84d4a328451783fe1fd48d3/users/ff4b9f161ff64a51b7e32f7f22c30289/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:58.127815 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76905b-415d-4263-814f-1a53219d15b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4261c49b84d4a328451783fe1fd48d3, user_id=ff4b9f161ff64a51b7e32f7f22c30289, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:58.130566 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76905b-415d-4263-814f-1a53219d15b8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:58.257170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0e826-9315-4a20-9fcc-1b6815d3fbd1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:58.259880 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c0e0e826-9315-4a20-9fcc-1b6815d3fbd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:58.270732 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1336/2685] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:58 2022] PUT /identity/v3/projects/f4261c49b84d4a328451783fe1fd48d3/users/ff4b9f161ff64a51b7e32f7f22c30289/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:58.275437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76905b-415d-4263-814f-1a53219d15b8 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:58.281609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0e826-9315-4a20-9fcc-1b6815d3fbd1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:58.353281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:58.354349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:58.354349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:58.354349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] PATH_INFO: `/v3/users/842e7dea2bb24389b900d29351a0bd33` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:58.355607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=842e7dea2bb24389b900d29351a0bd33)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:58.359138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:58.364845 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:58.369939 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76905b-415d-4263-814f-1a53219d15b8 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:58.377050 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd11804-2a05-4cf5-8c02-cb8d67f16b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:58.377621 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd11804-2a05-4cf5-8c02-cb8d67f16b03 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:58.377896 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd11804-2a05-4cf5-8c02-cb8d67f16b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:58.378692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd11804-2a05-4cf5-8c02-cb8d67f16b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:58.378848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd11804-2a05-4cf5-8c02-cb8d67f16b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:58.380454 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd11804-2a05-4cf5-8c02-cb8d67f16b03 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:58.391560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1337/2686] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:58 2022] GET /identity/v3/roles => generated 1427 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:58.397012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd11804-2a05-4cf5-8c02-cb8d67f16b03 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:58.408991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd11804-2a05-4cf5-8c02-cb8d67f16b03 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:58.413786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d60a1d-9c73-4723-9f78-cf2107efae04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:58.414445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d60a1d-9c73-4723-9f78-cf2107efae04 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:58.414719 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d60a1d-9c73-4723-9f78-cf2107efae04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:58.414988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d60a1d-9c73-4723-9f78-cf2107efae04 admin admin] PATH_INFO: `/v3/projects/f4261c49b84d4a328451783fe1fd48d3/users/ff4b9f161ff64a51b7e32f7f22c30289/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:58.416958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d60a1d-9c73-4723-9f78-cf2107efae04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4261c49b84d4a328451783fe1fd48d3, user_id=ff4b9f161ff64a51b7e32f7f22c30289, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:58.420246 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d60a1d-9c73-4723-9f78-cf2107efae04 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:58.433117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1338/2687] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:58 2022] PUT /identity/v3/projects/f4261c49b84d4a328451783fe1fd48d3/users/ff4b9f161ff64a51b7e32f7f22c30289/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:58.503329 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916951f-7fe0-4dc7-9366-d1219f2c0641 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:58.503615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916951f-7fe0-4dc7-9366-d1219f2c0641 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:58.503904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916951f-7fe0-4dc7-9366-d1219f2c0641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:58.532771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5916951f-7fe0-4dc7-9366-d1219f2c0641 None None] MFA Rules not processed for user `ff4b9f161ff64a51b7e32f7f22c30289`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:58.534007 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5916951f-7fe0-4dc7-9366-d1219f2c0641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:58.549512 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:58.609054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842e7dea2bb24389b900d29351a0bd33'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:58.609054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842e7dea2bb24389b900d29351a0bd33'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:58.609054 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842e7dea2bb24389b900d29351a0bd33'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:58.614492 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1339/2688] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:11:58.625682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916951f-7fe0-4dc7-9366-d1219f2c0641 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:58.627180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5916951f-7fe0-4dc7-9366-d1219f2c0641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:58.689651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916951f-7fe0-4dc7-9366-d1219f2c0641 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:58.694184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62b1f6-4cf0-489d-a374-db330c031c76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:58.701040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62b1f6-4cf0-489d-a374-db330c031c76 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:58.701040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62b1f6-4cf0-489d-a374-db330c031c76 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:58.701040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62b1f6-4cf0-489d-a374-db330c031c76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:58.701040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ac62b1f6-4cf0-489d-a374-db330c031c76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:58.720158 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842e7dea2bb24389b900d29351a0bd33'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:58.734511 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842e7dea2bb24389b900d29351a0bd33'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:58.748508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1350/2689] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:11:58 2022] DELETE /identity/v3/users/842e7dea2bb24389b900d29351a0bd33 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:58.759493 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62b1f6-4cf0-489d-a374-db330c031c76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:58.760948 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62b1f6-4cf0-489d-a374-db330c031c76 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:58.771322 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1340/2690] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:11:58 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:11:58.996666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:58.998292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:59.078642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3adc46-676d-4d51-8bbc-96cea4da26a3 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:59.083620 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:59.084353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:59.084642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:59.084915 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] PATH_INFO: `/v3/projects/d373da7992054abfae0564708529a973` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:59.086085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d373da7992054abfae0564708529a973)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:59.088053 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:59.238626 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd373da7992054abfae0564708529a973', 'request_id': 'req-057972b9-05c3-4ace-a971-343601c38fd3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:59.247926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd373da7992054abfae0564708529a973', 'request_id': 'req-057972b9-05c3-4ace-a971-343601c38fd3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:59.256355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd373da7992054abfae0564708529a973', 'request_id': 'req-057972b9-05c3-4ace-a971-343601c38fd3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:59.321449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] The token cache is being invalidate because project d373da7992054abfae0564708529a973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:11:59.321927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:59.322955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-057972b9-05c3-4ace-a971-343601c38fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:11:59.324773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1351/2691] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:11:58 2022] DELETE /identity/v3/projects/d373da7992054abfae0564708529a973 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:11:59.331360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb0e8-71f7-40ae-a3c8-71c6feb22c83 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:59.331900 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb0e8-71f7-40ae-a3c8-71c6feb22c83 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:59.332000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debb0e8-71f7-40ae-a3c8-71c6feb22c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:59.352610 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-6debb0e8-71f7-40ae-a3c8-71c6feb22c83 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:59.353810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6debb0e8-71f7-40ae-a3c8-71c6feb22c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:59.423468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1341/2692] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:11:59.432027 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a262c7-de2c-41e1-acc1-52a6c00c22ae None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:59.432574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a262c7-de2c-41e1-acc1-52a6c00c22ae None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:59.432574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a262c7-de2c-41e1-acc1-52a6c00c22ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:59.455830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-10a262c7-de2c-41e1-acc1-52a6c00c22ae None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:11:59.457106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-10a262c7-de2c-41e1-acc1-52a6c00c22ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:11:59.466732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1352/2693] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:11:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:11:59.472135 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debb0e8-71f7-40ae-a3c8-71c6feb22c83 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:59.490278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debb0e8-71f7-40ae-a3c8-71c6feb22c83 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:59.498499 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce05b5e-2039-4ed1-805d-306a82908097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:59.499578 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce05b5e-2039-4ed1-805d-306a82908097 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:59.500104 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce05b5e-2039-4ed1-805d-306a82908097 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:59.500517 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce05b5e-2039-4ed1-805d-306a82908097 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:59.501520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce05b5e-2039-4ed1-805d-306a82908097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:11:59.501970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce05b5e-2039-4ed1-805d-306a82908097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:59.504813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce05b5e-2039-4ed1-805d-306a82908097 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:59.507310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=100 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:59.528049 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1342/2694] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:11:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:11:59.529712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a262c7-de2c-41e1-acc1-52a6c00c22ae None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:59.543961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a262c7-de2c-41e1-acc1-52a6c00c22ae None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:59.550740 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6a1f9-7054-4d1e-9697-f7c093126140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:59.551692 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6a1f9-7054-4d1e-9697-f7c093126140 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:59.552125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6a1f9-7054-4d1e-9697-f7c093126140 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:59.552622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6a1f9-7054-4d1e-9697-f7c093126140 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:59.553894 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6a1f9-7054-4d1e-9697-f7c093126140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:59.557116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6a1f9-7054-4d1e-9697-f7c093126140 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:59.566207 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:11:59.651873 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1353/2695] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:11:59 2022] POST /identity/v3/projects => generated 406 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:11:59.653285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce05b5e-2039-4ed1-805d-306a82908097 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:59.664020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce05b5e-2039-4ed1-805d-306a82908097 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:59.728438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa540f74-72f9-4cc1-8fa3-e0e2aa0d42aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:59.730920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa540f74-72f9-4cc1-8fa3-e0e2aa0d42aa admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:59.730920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa540f74-72f9-4cc1-8fa3-e0e2aa0d42aa admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:59.730920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa540f74-72f9-4cc1-8fa3-e0e2aa0d42aa admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:59.730920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa540f74-72f9-4cc1-8fa3-e0e2aa0d42aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:59.733448 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa540f74-72f9-4cc1-8fa3-e0e2aa0d42aa admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:59.833741 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1343/2696] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:11:59 2022] POST /identity/v3/users => generated 304 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:11:59.838354 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6a1f9-7054-4d1e-9697-f7c093126140 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:59.850691 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6a1f9-7054-4d1e-9697-f7c093126140 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:59.857732 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de00c81-29d9-430f-b8d0-6539acc4ab00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:59.858673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de00c81-29d9-430f-b8d0-6539acc4ab00 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:59.859051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de00c81-29d9-430f-b8d0-6539acc4ab00 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:59.859425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de00c81-29d9-430f-b8d0-6539acc4ab00 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:59.860255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de00c81-29d9-430f-b8d0-6539acc4ab00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:59.862389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de00c81-29d9-430f-b8d0-6539acc4ab00 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:59.875573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1354/2697] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:59 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:59.879762 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa540f74-72f9-4cc1-8fa3-e0e2aa0d42aa admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:59.890498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa540f74-72f9-4cc1-8fa3-e0e2aa0d42aa admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:59.901081 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa37753-cb03-42e7-a2ad-1f3a47866ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:59.901766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa37753-cb03-42e7-a2ad-1f3a47866ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:59.902082 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa37753-cb03-42e7-a2ad-1f3a47866ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:59.902401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa37753-cb03-42e7-a2ad-1f3a47866ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:59.903310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa37753-cb03-42e7-a2ad-1f3a47866ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:59.905040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa37753-cb03-42e7-a2ad-1f3a47866ab5 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:11:59.916797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1344/2698] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:11:59 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:11:59.921616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de00c81-29d9-430f-b8d0-6539acc4ab00 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:11:59.933132 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de00c81-29d9-430f-b8d0-6539acc4ab00 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:11:59.938889 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016705eb-92cd-49f5-917c-e3a1fc96a416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:11:59.939637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016705eb-92cd-49f5-917c-e3a1fc96a416 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:11:59.939918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016705eb-92cd-49f5-917c-e3a1fc96a416 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:11:59.940192 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016705eb-92cd-49f5-917c-e3a1fc96a416 admin admin] PATH_INFO: `/v3/projects/356c4ce77a07443b829b031eb9c46a6d/users/85d17fe5a97c46938577e176097e63f2/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:11:59.956521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016705eb-92cd-49f5-917c-e3a1fc96a416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=356c4ce77a07443b829b031eb9c46a6d, user_id=85d17fe5a97c46938577e176097e63f2, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:11:59.959236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016705eb-92cd-49f5-917c-e3a1fc96a416 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:00.078974 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1355/2699] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:11:59 2022] PUT /identity/v3/projects/356c4ce77a07443b829b031eb9c46a6d/users/85d17fe5a97c46938577e176097e63f2/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:12:00.094239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c1ba1-4376-4865-8ebc-6cbd065badbb None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:00.094521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c1ba1-4376-4865-8ebc-6cbd065badbb None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:00.094792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c1ba1-4376-4865-8ebc-6cbd065badbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:00.125051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5d0c1ba1-4376-4865-8ebc-6cbd065badbb None None] MFA Rules not processed for user `85d17fe5a97c46938577e176097e63f2`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:00.126284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5d0c1ba1-4376-4865-8ebc-6cbd065badbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:00.206295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1345/2700] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:00 2022] POST /identity/v3/auth/tokens => generated 3292 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:00.211063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016705eb-92cd-49f5-917c-e3a1fc96a416 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:00.284703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016705eb-92cd-49f5-917c-e3a1fc96a416 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:00.289604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c393bdd-87f1-445e-9125-0ce3a83f76f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:00.290211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c393bdd-87f1-445e-9125-0ce3a83f76f3 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:00.290488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c393bdd-87f1-445e-9125-0ce3a83f76f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:00.290758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c393bdd-87f1-445e-9125-0ce3a83f76f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:00.291429 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c393bdd-87f1-445e-9125-0ce3a83f76f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:00.293418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c393bdd-87f1-445e-9125-0ce3a83f76f3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:00.408069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1356/2701] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:00 2022] POST /identity/v3/projects => generated 404 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:00.412088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c1ba1-4376-4865-8ebc-6cbd065badbb None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:00.423150 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c1ba1-4376-4865-8ebc-6cbd065badbb None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:00.490472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16dfc3-0be6-46d6-909e-71d2a76d6651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:00.497337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16dfc3-0be6-46d6-909e-71d2a76d6651 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:00.497337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16dfc3-0be6-46d6-909e-71d2a76d6651 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:00.497337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16dfc3-0be6-46d6-909e-71d2a76d6651 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:00.497337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16dfc3-0be6-46d6-909e-71d2a76d6651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:00.497337 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b16dfc3-0be6-46d6-909e-71d2a76d6651 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:00.504129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:00.590567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1346/2702] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:00 2022] POST /identity/v3/users => generated 309 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:00.594990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c393bdd-87f1-445e-9125-0ce3a83f76f3 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:00.605757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c393bdd-87f1-445e-9125-0ce3a83f76f3 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:00.611855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daadfb6-6c44-4422-8d81-24b9caa50d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:00.612440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daadfb6-6c44-4422-8d81-24b9caa50d17 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:00.612716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daadfb6-6c44-4422-8d81-24b9caa50d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:00.613318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daadfb6-6c44-4422-8d81-24b9caa50d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:00.613613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daadfb6-6c44-4422-8d81-24b9caa50d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:00.615235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daadfb6-6c44-4422-8d81-24b9caa50d17 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:00.628386 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1357/2703] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:00 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:00.633221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16dfc3-0be6-46d6-909e-71d2a76d6651 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:00.643326 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b16dfc3-0be6-46d6-909e-71d2a76d6651 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:00.648099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffeb8c3-1c9a-4a6e-9118-4b0e6c4d1524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:00.648754 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffeb8c3-1c9a-4a6e-9118-4b0e6c4d1524 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:00.649026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffeb8c3-1c9a-4a6e-9118-4b0e6c4d1524 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:00.649296 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffeb8c3-1c9a-4a6e-9118-4b0e6c4d1524 admin admin] PATH_INFO: `/v3/projects/d0a0ac94f6c54911865b6a2d012eb1db/users/dbe42e7d4eff4dc3933ce8ab7dfbc595/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:00.665427 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffeb8c3-1c9a-4a6e-9118-4b0e6c4d1524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0a0ac94f6c54911865b6a2d012eb1db, user_id=dbe42e7d4eff4dc3933ce8ab7dfbc595, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:00.672581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffeb8c3-1c9a-4a6e-9118-4b0e6c4d1524 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:00.784873 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1347/2704] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:00 2022] PUT /identity/v3/projects/d0a0ac94f6c54911865b6a2d012eb1db/users/dbe42e7d4eff4dc3933ce8ab7dfbc595/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:00.789400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daadfb6-6c44-4422-8d81-24b9caa50d17 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:00.800439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daadfb6-6c44-4422-8d81-24b9caa50d17 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:00.867324 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d273e0-e7b8-4c6c-8799-4fba675516a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:00.867970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d273e0-e7b8-4c6c-8799-4fba675516a6 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:00.868248 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d273e0-e7b8-4c6c-8799-4fba675516a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:00.868520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d273e0-e7b8-4c6c-8799-4fba675516a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:00.869163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d273e0-e7b8-4c6c-8799-4fba675516a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:00.870784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d273e0-e7b8-4c6c-8799-4fba675516a6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:00.881565 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1358/2705] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:00 2022] GET /identity/v3/roles => generated 1427 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:12:00.885856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffeb8c3-1c9a-4a6e-9118-4b0e6c4d1524 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:00.896749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffeb8c3-1c9a-4a6e-9118-4b0e6c4d1524 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:00.902560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c60e3e-3133-4b9a-bfb1-86a7f98d373d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:00.903782 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c60e3e-3133-4b9a-bfb1-86a7f98d373d admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:00.904059 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c60e3e-3133-4b9a-bfb1-86a7f98d373d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:00.904422 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c60e3e-3133-4b9a-bfb1-86a7f98d373d admin admin] PATH_INFO: `/v3/projects/d0a0ac94f6c54911865b6a2d012eb1db/users/dbe42e7d4eff4dc3933ce8ab7dfbc595/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:00.906346 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c60e3e-3133-4b9a-bfb1-86a7f98d373d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0a0ac94f6c54911865b6a2d012eb1db, user_id=dbe42e7d4eff4dc3933ce8ab7dfbc595, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:00.909079 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c60e3e-3133-4b9a-bfb1-86a7f98d373d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:00.920587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1348/2706] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:00 2022] PUT /identity/v3/projects/d0a0ac94f6c54911865b6a2d012eb1db/users/dbe42e7d4eff4dc3933ce8ab7dfbc595/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:00.931645 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927069f-1596-43a0-bfa6-256c58b05d99 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:00.931934 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927069f-1596-43a0-bfa6-256c58b05d99 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:00.932211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1927069f-1596-43a0-bfa6-256c58b05d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:00.964209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-1927069f-1596-43a0-bfa6-256c58b05d99 None None] MFA Rules not processed for user `dbe42e7d4eff4dc3933ce8ab7dfbc595`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:00.966168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1927069f-1596-43a0-bfa6-256c58b05d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:00.976584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:01.040483 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1359/2707] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:00 2022] POST /identity/v3/auth/tokens => generated 3357 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:01.052281 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c60e3e-3133-4b9a-bfb1-86a7f98d373d admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:01.054240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b9c60e3e-3133-4b9a-bfb1-86a7f98d373d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:01.123208 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c60e3e-3133-4b9a-bfb1-86a7f98d373d admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:01.127371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71a3c0-5dc8-4182-a594-2f0f94f01dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:01.127992 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71a3c0-5dc8-4182-a594-2f0f94f01dee service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:01.128263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71a3c0-5dc8-4182-a594-2f0f94f01dee service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:01.128533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71a3c0-5dc8-4182-a594-2f0f94f01dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:01.138147 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71a3c0-5dc8-4182-a594-2f0f94f01dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:01.139606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71a3c0-5dc8-4182-a594-2f0f94f01dee service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:01.149445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1349/2708] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:01 2022] GET /identity/v3/auth/tokens => generated 3357 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:01.296255 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927069f-1596-43a0-bfa6-256c58b05d99 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:01.366545 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1927069f-1596-43a0-bfa6-256c58b05d99 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:01.372017 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:01.373656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:01.373985 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:01.374263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] PATH_INFO: `/v3/users/85d17fe5a97c46938577e176097e63f2` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:01.375498 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d17fe5a97c46938577e176097e63f2)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:01.377422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:01.608318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d17fe5a97c46938577e176097e63f2'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:01.609127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d17fe5a97c46938577e176097e63f2'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:01.609777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d17fe5a97c46938577e176097e63f2'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:01.740354 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d17fe5a97c46938577e176097e63f2'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:01.761418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d17fe5a97c46938577e176097e63f2'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:01.793018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1360/2709] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:01 2022] DELETE /identity/v3/users/85d17fe5a97c46938577e176097e63f2 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:01.794894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53049464-8f67-4756-89f1-8493cb5142fa None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:01.795411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53049464-8f67-4756-89f1-8493cb5142fa None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:01.795513 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53049464-8f67-4756-89f1-8493cb5142fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:01.816725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-53049464-8f67-4756-89f1-8493cb5142fa None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:01.817839 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-53049464-8f67-4756-89f1-8493cb5142fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:01.864713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:01.866160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:01.892462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1350/2710] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:01.903319 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:01.909864 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53049464-8f67-4756-89f1-8493cb5142fa None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:01.911363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-53049464-8f67-4756-89f1-8493cb5142fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:01.937808 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7bf915-de5a-4cf2-9244-5a0d8ae2ee25 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:01.941790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f39ec-01fb-41ab-aa3d-59a5d76a683d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:01.942408 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f39ec-01fb-41ab-aa3d-59a5d76a683d service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:01.942687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f39ec-01fb-41ab-aa3d-59a5d76a683d service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:01.942961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f39ec-01fb-41ab-aa3d-59a5d76a683d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:01.945118 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c01f39ec-01fb-41ab-aa3d-59a5d76a683d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:01.988378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53049464-8f67-4756-89f1-8493cb5142fa None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:01.992136 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a4138-3eac-412d-bf24-14473437841d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:01.992766 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a4138-3eac-412d-bf24-14473437841d service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:01.993037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a4138-3eac-412d-bf24-14473437841d service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:01.993300 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a4138-3eac-412d-bf24-14473437841d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:02.013102 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a4138-3eac-412d-bf24-14473437841d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:02.014837 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a4138-3eac-412d-bf24-14473437841d service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:02.015269 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f39ec-01fb-41ab-aa3d-59a5d76a683d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:02.017069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f39ec-01fb-41ab-aa3d-59a5d76a683d service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:02.023495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:02.028418 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1361/2711] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:12:02.029615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1351/2712] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:01 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:12:02.466571 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f39ec-01fb-41ab-aa3d-59a5d76a683d service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:02.468230 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c01f39ec-01fb-41ab-aa3d-59a5d76a683d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:02.484926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f39ec-01fb-41ab-aa3d-59a5d76a683d service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:02.490058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:02.492878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:02.492878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:02.492878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] PATH_INFO: `/v3/projects/356c4ce77a07443b829b031eb9c46a6d` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:02.493353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=356c4ce77a07443b829b031eb9c46a6d)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:02.495055 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:02.639775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '356c4ce77a07443b829b031eb9c46a6d', 'request_id': 'req-59803fe9-0406-44ee-a6c7-aab2648c77fd'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:02.646084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '356c4ce77a07443b829b031eb9c46a6d', 'request_id': 'req-59803fe9-0406-44ee-a6c7-aab2648c77fd'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:02.652303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '356c4ce77a07443b829b031eb9c46a6d', 'request_id': 'req-59803fe9-0406-44ee-a6c7-aab2648c77fd'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:02.740124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] The token cache is being invalidate because project 356c4ce77a07443b829b031eb9c46a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:02.740439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:02.741094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:02.742659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1362/2713] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:02 2022] DELETE /identity/v3/projects/356c4ce77a07443b829b031eb9c46a6d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:02.746916 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a4138-3eac-412d-bf24-14473437841d service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:02.756871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a4138-3eac-412d-bf24-14473437841d service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:02.825465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:02.826655 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:02.827127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:02.827576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] PATH_INFO: `/v3/users/dbe42e7d4eff4dc3933ce8ab7dfbc595` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:02.829407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbe42e7d4eff4dc3933ce8ab7dfbc595)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:02.832822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:03.064130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbe42e7d4eff4dc3933ce8ab7dfbc595'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:03.064817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbe42e7d4eff4dc3933ce8ab7dfbc595'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:03.065450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbe42e7d4eff4dc3933ce8ab7dfbc595'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:03.206356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbe42e7d4eff4dc3933ce8ab7dfbc595'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:03.221553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbe42e7d4eff4dc3933ce8ab7dfbc595'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:03.234272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1352/2714] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:02 2022] DELETE /identity/v3/users/dbe42e7d4eff4dc3933ce8ab7dfbc595 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:03.662931 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:03.664575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:03.681120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59803fe9-0406-44ee-a6c7-aab2648c77fd admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:03.752490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:03.753292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:03.753637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:03.754325 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] PATH_INFO: `/v3/projects/d0a0ac94f6c54911865b6a2d012eb1db` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:03.755168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0a0ac94f6c54911865b6a2d012eb1db)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:03.757182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:03.768234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:03.769764 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:03.772582 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:03.844228 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9cc7f-15c1-4dd9-a02d-427ab50e0ff8 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:03.848030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bbcd6c-88ec-4e04-871b-ee96718b4200 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:03.848644 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbcd6c-88ec-4e04-871b-ee96718b4200 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:03.848932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbcd6c-88ec-4e04-871b-ee96718b4200 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:03.849200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbcd6c-88ec-4e04-871b-ee96718b4200 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:03.850961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-08bbcd6c-88ec-4e04-871b-ee96718b4200 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:03.889755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:03.911710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0a0ac94f6c54911865b6a2d012eb1db', 'request_id': 'req-93eeaa9e-132b-45fb-b205-7bb3f58877c3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:03.921264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0a0ac94f6c54911865b6a2d012eb1db', 'request_id': 'req-93eeaa9e-132b-45fb-b205-7bb3f58877c3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:03.927606 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0a0ac94f6c54911865b6a2d012eb1db', 'request_id': 'req-93eeaa9e-132b-45fb-b205-7bb3f58877c3'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:03.932133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbcd6c-88ec-4e04-871b-ee96718b4200 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:03.936658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bbcd6c-88ec-4e04-871b-ee96718b4200 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:03.945453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1353/2715] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:04.000270 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] The token cache is being invalidate because project d0a0ac94f6c54911865b6a2d012eb1db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:04.000688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:04.001344 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-93eeaa9e-132b-45fb-b205-7bb3f58877c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:04.002867 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1363/2716] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:03 2022] DELETE /identity/v3/projects/d0a0ac94f6c54911865b6a2d012eb1db => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:04.007749 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08fb54-e248-4d26-9936-cb74ecfe3552 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:04.008034 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08fb54-e248-4d26-9936-cb74ecfe3552 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:04.008335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08fb54-e248-4d26-9936-cb74ecfe3552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:04.029502 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-2a08fb54-e248-4d26-9936-cb74ecfe3552 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:04.030786 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2a08fb54-e248-4d26-9936-cb74ecfe3552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:04.106060 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1354/2717] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:04.115852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc350a9-6591-4a22-9b14-5a3c073e3339 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:04.116142 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc350a9-6591-4a22-9b14-5a3c073e3339 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:04.116420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc350a9-6591-4a22-9b14-5a3c073e3339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:04.137141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-6dc350a9-6591-4a22-9b14-5a3c073e3339 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:04.138544 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-6dc350a9-6591-4a22-9b14-5a3c073e3339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:04.146695 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1364/2718] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:04.151098 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08fb54-e248-4d26-9936-cb74ecfe3552 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:04.161562 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08fb54-e248-4d26-9936-cb74ecfe3552 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:04.166285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c005a-6f97-4d60-8cc8-17daff70b9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:04.172905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c005a-6f97-4d60-8cc8-17daff70b9eb admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:04.172905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c005a-6f97-4d60-8cc8-17daff70b9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:04.172905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c005a-6f97-4d60-8cc8-17daff70b9eb admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:04.172905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c005a-6f97-4d60-8cc8-17daff70b9eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:12:04.172905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c005a-6f97-4d60-8cc8-17daff70b9eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:04.172905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c005a-6f97-4d60-8cc8-17daff70b9eb admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:04.183416 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1355/2719] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:12:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:12:04.188391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc350a9-6591-4a22-9b14-5a3c073e3339 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:04.198238 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc350a9-6591-4a22-9b14-5a3c073e3339 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:04.202979 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb4771-f640-41f2-8c1b-5a09f96a5e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:04.203613 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb4771-f640-41f2-8c1b-5a09f96a5e6b admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:04.203896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb4771-f640-41f2-8c1b-5a09f96a5e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:04.204163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb4771-f640-41f2-8c1b-5a09f96a5e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:04.204829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb4771-f640-41f2-8c1b-5a09f96a5e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:04.206716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb4771-f640-41f2-8c1b-5a09f96a5e6b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:04.396967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1365/2720] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:04 2022] POST /identity/v3/projects => generated 374 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:04.401552 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c005a-6f97-4d60-8cc8-17daff70b9eb admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:04.412451 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c005a-6f97-4d60-8cc8-17daff70b9eb admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:04.473354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ee818-2361-4d91-b353-f0f307b98641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:04.474036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ee818-2361-4d91-b353-f0f307b98641 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:04.474324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ee818-2361-4d91-b353-f0f307b98641 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:04.474599 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ee818-2361-4d91-b353-f0f307b98641 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:04.475283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ee818-2361-4d91-b353-f0f307b98641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:04.477285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462ee818-2361-4d91-b353-f0f307b98641 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:04.564926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1356/2721] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:04 2022] POST /identity/v3/users => generated 288 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:04.569823 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb4771-f640-41f2-8c1b-5a09f96a5e6b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:04.581348 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb4771-f640-41f2-8c1b-5a09f96a5e6b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:04.587007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de455a95-45c6-4b82-921b-b6c24be4c27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:04.587632 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455a95-45c6-4b82-921b-b6c24be4c27b admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:04.587906 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455a95-45c6-4b82-921b-b6c24be4c27b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:04.588173 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de455a95-45c6-4b82-921b-b6c24be4c27b admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:04.588805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de455a95-45c6-4b82-921b-b6c24be4c27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:04.590413 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de455a95-45c6-4b82-921b-b6c24be4c27b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:04.601680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1366/2722] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:04 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:04.606221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ee818-2361-4d91-b353-f0f307b98641 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:04.617706 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462ee818-2361-4d91-b353-f0f307b98641 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:04.622758 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c67fd-f188-48b4-8b83-ad16939b0cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:04.623408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c67fd-f188-48b4-8b83-ad16939b0cae admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:04.623685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c67fd-f188-48b4-8b83-ad16939b0cae admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:04.623950 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c67fd-f188-48b4-8b83-ad16939b0cae admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:04.624593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c67fd-f188-48b4-8b83-ad16939b0cae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:04.626180 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c67fd-f188-48b4-8b83-ad16939b0cae admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:04.638261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1357/2723] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:04 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:04.643387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de455a95-45c6-4b82-921b-b6c24be4c27b admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:04.654802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de455a95-45c6-4b82-921b-b6c24be4c27b admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:04.659825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a2762-3680-4286-9e80-187169d5ac61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:04.660470 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a2762-3680-4286-9e80-187169d5ac61 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:04.660778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a2762-3680-4286-9e80-187169d5ac61 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:04.661046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a2762-3680-4286-9e80-187169d5ac61 admin admin] PATH_INFO: `/v3/projects/02dfe2d27de344f0824bfafa6be56906/users/2d79bee911b9455d86b6a0f096eae1cf/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:04.676901 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a2762-3680-4286-9e80-187169d5ac61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02dfe2d27de344f0824bfafa6be56906, user_id=2d79bee911b9455d86b6a0f096eae1cf, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:04.680072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a2762-3680-4286-9e80-187169d5ac61 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:04.750038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1367/2724] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:04 2022] PUT /identity/v3/projects/02dfe2d27de344f0824bfafa6be56906/users/2d79bee911b9455d86b6a0f096eae1cf/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:04.760522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f704773-8646-4ce0-9ca5-c7f2033c5f70 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:04.760893 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f704773-8646-4ce0-9ca5-c7f2033c5f70 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:04.761257 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f704773-8646-4ce0-9ca5-c7f2033c5f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:04.794658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-1f704773-8646-4ce0-9ca5-c7f2033c5f70 None None] MFA Rules not processed for user `2d79bee911b9455d86b6a0f096eae1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:04.795912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-1f704773-8646-4ce0-9ca5-c7f2033c5f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:04.821250 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:04.863897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1358/2725] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:04.868479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a2762-3680-4286-9e80-187169d5ac61 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:04.937967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a2762-3680-4286-9e80-187169d5ac61 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:04.944041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5b68b-104d-4165-9efd-2a9855dc729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:04.944041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5b68b-104d-4165-9efd-2a9855dc729c admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:04.944041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5b68b-104d-4165-9efd-2a9855dc729c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:04.944041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5b68b-104d-4165-9efd-2a9855dc729c admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:04.944733 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5b68b-104d-4165-9efd-2a9855dc729c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:04.946647 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5b68b-104d-4165-9efd-2a9855dc729c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:05.085894 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1368/2726] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:04 2022] POST /identity/v3/projects => generated 372 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:05.090308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f704773-8646-4ce0-9ca5-c7f2033c5f70 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:05.101746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f704773-8646-4ce0-9ca5-c7f2033c5f70 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:05.174544 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb6ace-d094-431a-ae9c-c13dd7659b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:05.175117 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb6ace-d094-431a-ae9c-c13dd7659b08 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:05.175395 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb6ace-d094-431a-ae9c-c13dd7659b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:05.175663 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb6ace-d094-431a-ae9c-c13dd7659b08 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:05.176315 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb6ace-d094-431a-ae9c-c13dd7659b08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:05.178563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb6ace-d094-431a-ae9c-c13dd7659b08 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:05.273399 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1359/2727] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:05 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:05.277603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5b68b-104d-4165-9efd-2a9855dc729c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:05.288387 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5b68b-104d-4165-9efd-2a9855dc729c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:05.294254 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a9fd8-6a5f-49af-9f84-7ce930d5bfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:05.295262 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9fd8-6a5f-49af-9f84-7ce930d5bfc9 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:05.295689 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9fd8-6a5f-49af-9f84-7ce930d5bfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:05.296138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9fd8-6a5f-49af-9f84-7ce930d5bfc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:05.297212 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a9fd8-6a5f-49af-9f84-7ce930d5bfc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:05.300095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a9fd8-6a5f-49af-9f84-7ce930d5bfc9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:05.313719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1369/2728] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:05 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:05.318056 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb6ace-d094-431a-ae9c-c13dd7659b08 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:05.328514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb6ace-d094-431a-ae9c-c13dd7659b08 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:05.333666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63bc08-376a-4c6f-b973-421a49204155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:05.334131 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63bc08-376a-4c6f-b973-421a49204155 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:05.334411 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63bc08-376a-4c6f-b973-421a49204155 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:05.334686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a63bc08-376a-4c6f-b973-421a49204155 admin admin] PATH_INFO: `/v3/projects/8a5b9b027eea418292e35bb28ffca752/users/a96c6f6f07f84720bc443da7c2cdb37d/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:05.360851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63bc08-376a-4c6f-b973-421a49204155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5b9b027eea418292e35bb28ffca752, user_id=a96c6f6f07f84720bc443da7c2cdb37d, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:05.360851 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a63bc08-376a-4c6f-b973-421a49204155 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:05.480915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1360/2729] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:05 2022] PUT /identity/v3/projects/8a5b9b027eea418292e35bb28ffca752/users/a96c6f6f07f84720bc443da7c2cdb37d/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:05.485628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a9fd8-6a5f-49af-9f84-7ce930d5bfc9 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:05.495748 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:05.501004 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a9fd8-6a5f-49af-9f84-7ce930d5bfc9 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:05.568967 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72551154-17bb-4732-b1d3-0551a3074bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:05.569581 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72551154-17bb-4732-b1d3-0551a3074bac admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:05.569860 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72551154-17bb-4732-b1d3-0551a3074bac admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:05.570138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72551154-17bb-4732-b1d3-0551a3074bac admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:05.570781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72551154-17bb-4732-b1d3-0551a3074bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:05.572445 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72551154-17bb-4732-b1d3-0551a3074bac admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:05.583700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1370/2730] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:05 2022] GET /identity/v3/roles => generated 1427 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:05.588071 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63bc08-376a-4c6f-b973-421a49204155 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:05.599388 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a63bc08-376a-4c6f-b973-421a49204155 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:05.607162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd465e-f1c7-4052-b563-9c5e7e561fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:05.607162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd465e-f1c7-4052-b563-9c5e7e561fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:05.607162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd465e-f1c7-4052-b563-9c5e7e561fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:05.607162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd465e-f1c7-4052-b563-9c5e7e561fa7 admin admin] PATH_INFO: `/v3/projects/8a5b9b027eea418292e35bb28ffca752/users/a96c6f6f07f84720bc443da7c2cdb37d/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:05.607621 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd465e-f1c7-4052-b563-9c5e7e561fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5b9b027eea418292e35bb28ffca752, user_id=a96c6f6f07f84720bc443da7c2cdb37d, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:05.610292 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd465e-f1c7-4052-b563-9c5e7e561fa7 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:05.623051 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1361/2731] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:05 2022] PUT /identity/v3/projects/8a5b9b027eea418292e35bb28ffca752/users/a96c6f6f07f84720bc443da7c2cdb37d/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:05.634151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e8abb-3ba5-4591-ac59-fdd96f02d6fb None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:05.634530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e8abb-3ba5-4591-ac59-fdd96f02d6fb None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:05.634835 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6e8abb-3ba5-4591-ac59-fdd96f02d6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:05.664642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-4d6e8abb-3ba5-4591-ac59-fdd96f02d6fb None None] MFA Rules not processed for user `a96c6f6f07f84720bc443da7c2cdb37d`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:05.665909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4d6e8abb-3ba5-4591-ac59-fdd96f02d6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:05.739097 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1371/2732] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:05.751882 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd465e-f1c7-4052-b563-9c5e7e561fa7 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:05.753651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b7fd465e-f1c7-4052-b563-9c5e7e561fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:05.824808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd465e-f1c7-4052-b563-9c5e7e561fa7 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:05.829269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7facc4-723c-4a73-b5aa-221ccb5c31bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:05.829876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7facc4-723c-4a73-b5aa-221ccb5c31bf service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:05.830197 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7facc4-723c-4a73-b5aa-221ccb5c31bf service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:05.830464 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7facc4-723c-4a73-b5aa-221ccb5c31bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:05.839982 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7facc4-723c-4a73-b5aa-221ccb5c31bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:05.842158 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7facc4-723c-4a73-b5aa-221ccb5c31bf service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:05.852262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1362/2733] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:06.970449 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e8abb-3ba5-4591-ac59-fdd96f02d6fb None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:07.034401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6e8abb-3ba5-4591-ac59-fdd96f02d6fb None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:07.039102 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:07.039731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:07.040003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:07.040268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] PATH_INFO: `/v3/users/2d79bee911b9455d86b6a0f096eae1cf` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:07.041437 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d79bee911b9455d86b6a0f096eae1cf)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:07.043756 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:07.151326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:07.270065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d79bee911b9455d86b6a0f096eae1cf'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:07.270874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d79bee911b9455d86b6a0f096eae1cf'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:07.271414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d79bee911b9455d86b6a0f096eae1cf'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:07.400658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d79bee911b9455d86b6a0f096eae1cf'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:07.414830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d79bee911b9455d86b6a0f096eae1cf'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:07.428943 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1372/2734] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:06 2022] DELETE /identity/v3/users/2d79bee911b9455d86b6a0f096eae1cf => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:07.434014 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3531e42-3a87-4269-a328-cd6da2ddf868 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:07.436118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3531e42-3a87-4269-a328-cd6da2ddf868 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:07.436118 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3531e42-3a87-4269-a328-cd6da2ddf868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:07.454535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-a3531e42-3a87-4269-a328-cd6da2ddf868 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:07.455811 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a3531e42-3a87-4269-a328-cd6da2ddf868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:07.472929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:07.519695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1363/2735] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:07.532798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:07.534393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:07.594549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7899606-a20d-4bad-b31a-9c433244c8ff admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:07.598374 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d672c-f054-48c7-a98a-696cf96a28db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:07.598978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d672c-f054-48c7-a98a-696cf96a28db service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:07.599315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d672c-f054-48c7-a98a-696cf96a28db service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:07.599586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d672c-f054-48c7-a98a-696cf96a28db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:07.613512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d672c-f054-48c7-a98a-696cf96a28db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:07.619573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862d672c-f054-48c7-a98a-696cf96a28db service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:07.624992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1373/2736] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:08.070880 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3531e42-3a87-4269-a328-cd6da2ddf868 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:08.072401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a3531e42-3a87-4269-a328-cd6da2ddf868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:08.086251 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3531e42-3a87-4269-a328-cd6da2ddf868 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:08.091215 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:08.091861 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:08.098414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:08.098414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] PATH_INFO: `/v3/projects/02dfe2d27de344f0824bfafa6be56906` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:08.098414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02dfe2d27de344f0824bfafa6be56906)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:08.098414 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:08.246433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02dfe2d27de344f0824bfafa6be56906', 'request_id': 'req-a62fddaf-b836-4176-b9b0-97ab3592e491'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:08.254010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02dfe2d27de344f0824bfafa6be56906', 'request_id': 'req-a62fddaf-b836-4176-b9b0-97ab3592e491'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:08.349227 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02dfe2d27de344f0824bfafa6be56906', 'request_id': 'req-a62fddaf-b836-4176-b9b0-97ab3592e491'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:08.362210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] The token cache is being invalidate because project 02dfe2d27de344f0824bfafa6be56906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:08.362499 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:08.363221 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:08.364917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1364/2737] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:08 2022] DELETE /identity/v3/projects/02dfe2d27de344f0824bfafa6be56906 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:08.369167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d672c-f054-48c7-a98a-696cf96a28db service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:08.388058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d672c-f054-48c7-a98a-696cf96a28db service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:08.462778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:08.462778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:08.462778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:08.462778 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] PATH_INFO: `/v3/users/a96c6f6f07f84720bc443da7c2cdb37d` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:08.463190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a96c6f6f07f84720bc443da7c2cdb37d)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:08.463190 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:08.728806 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a96c6f6f07f84720bc443da7c2cdb37d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:08.729420 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a96c6f6f07f84720bc443da7c2cdb37d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:08.730152 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a96c6f6f07f84720bc443da7c2cdb37d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:08.733483 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:08.861144 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a96c6f6f07f84720bc443da7c2cdb37d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:08.875296 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-15bc2785-9e31-41f8-9499-46dadba11d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a96c6f6f07f84720bc443da7c2cdb37d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:08.887792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1374/2738] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:08 2022] DELETE /identity/v3/users/a96c6f6f07f84720bc443da7c2cdb37d => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:09.397303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:09.397303 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:09.405347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fddaf-b836-4176-b9b0-97ab3592e491 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:09.468083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:09.468845 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:09.469125 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:09.469397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] PATH_INFO: `/v3/projects/8a5b9b027eea418292e35bb28ffca752` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:09.470640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5b9b027eea418292e35bb28ffca752)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:09.472564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:09.619977 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:09.623298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5b9b027eea418292e35bb28ffca752', 'request_id': 'req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:09.629604 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5b9b027eea418292e35bb28ffca752', 'request_id': 'req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:09.720685 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5b9b027eea418292e35bb28ffca752', 'request_id': 'req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:09.732269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] The token cache is being invalidate because project 8a5b9b027eea418292e35bb28ffca752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:09.732574 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:09.733261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-25ac1fd1-802a-4932-bd51-cfa5d9d26ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:09.734756 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1365/2739] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:09 2022] DELETE /identity/v3/projects/8a5b9b027eea418292e35bb28ffca752 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:09.741046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164c29e-25b2-4741-bc47-694512f94ed9 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:09.741570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164c29e-25b2-4741-bc47-694512f94ed9 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:09.741945 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164c29e-25b2-4741-bc47-694512f94ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:09.765918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-1164c29e-25b2-4741-bc47-694512f94ed9 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:09.767160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1164c29e-25b2-4741-bc47-694512f94ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:09.834064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1375/2740] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:09.843700 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bef85e-5c49-4c60-a8c9-152b4124322c None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:09.843983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bef85e-5c49-4c60-a8c9-152b4124322c None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:09.844255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bef85e-5c49-4c60-a8c9-152b4124322c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:09.864255 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b4bef85e-5c49-4c60-a8c9-152b4124322c None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:09.865504 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b4bef85e-5c49-4c60-a8c9-152b4124322c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:09.873854 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1366/2741] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:09.878569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164c29e-25b2-4741-bc47-694512f94ed9 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:09.890272 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164c29e-25b2-4741-bc47-694512f94ed9 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:09.895327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3606ecd-9184-4218-8e44-8e5d0ede144e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:09.896030 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3606ecd-9184-4218-8e44-8e5d0ede144e admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:09.896312 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3606ecd-9184-4218-8e44-8e5d0ede144e admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:09.896566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3606ecd-9184-4218-8e44-8e5d0ede144e admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:09.897216 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3606ecd-9184-4218-8e44-8e5d0ede144e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:12:09.897518 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3606ecd-9184-4218-8e44-8e5d0ede144e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:09.899128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3606ecd-9184-4218-8e44-8e5d0ede144e admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:09.912124 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1376/2742] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:12:09 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:12:09.916727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bef85e-5c49-4c60-a8c9-152b4124322c None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:09.927540 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bef85e-5c49-4c60-a8c9-152b4124322c None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:09.933752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10273cb-dac3-4d1e-8189-9e5767088380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:09.934912 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10273cb-dac3-4d1e-8189-9e5767088380 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:09.935307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10273cb-dac3-4d1e-8189-9e5767088380 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:09.935482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10273cb-dac3-4d1e-8189-9e5767088380 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:09.936145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10273cb-dac3-4d1e-8189-9e5767088380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:09.938078 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10273cb-dac3-4d1e-8189-9e5767088380 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:10.032144 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1367/2743] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:09 2022] POST /identity/v3/projects => generated 364 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:10.036505 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3606ecd-9184-4218-8e44-8e5d0ede144e admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:10.047122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3606ecd-9184-4218-8e44-8e5d0ede144e admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:10.106521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060ac43-f6b1-4e9e-9546-53dade98f667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:10.107357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060ac43-f6b1-4e9e-9546-53dade98f667 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:10.107490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060ac43-f6b1-4e9e-9546-53dade98f667 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:10.107738 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060ac43-f6b1-4e9e-9546-53dade98f667 admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:10.108377 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060ac43-f6b1-4e9e-9546-53dade98f667 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:10.110309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060ac43-f6b1-4e9e-9546-53dade98f667 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:10.207767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1377/2744] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:10 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:10.211988 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10273cb-dac3-4d1e-8189-9e5767088380 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:10.217826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060ac43-f6b1-4e9e-9546-53dade98f667 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:10.219643 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c060ac43-f6b1-4e9e-9546-53dade98f667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:10.223119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10273cb-dac3-4d1e-8189-9e5767088380 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:10.227800 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754dec8-bb60-4bdc-bb28-fcea45aecbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:10.228412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754dec8-bb60-4bdc-bb28-fcea45aecbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:10.228691 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754dec8-bb60-4bdc-bb28-fcea45aecbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:10.228958 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754dec8-bb60-4bdc-bb28-fcea45aecbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:10.229595 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754dec8-bb60-4bdc-bb28-fcea45aecbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:10.231252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754dec8-bb60-4bdc-bb28-fcea45aecbc4 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:10.234992 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060ac43-f6b1-4e9e-9546-53dade98f667 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:10.238291 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:10.238982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:10.239292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:10.239566 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] PATH_INFO: `/v3/users/346b16eccc4e4535b3482e22e115249d` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:10.240781 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=346b16eccc4e4535b3482e22e115249d)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:10.242439 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1368/2745] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:10 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:10.243147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:10.248657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754dec8-bb60-4bdc-bb28-fcea45aecbc4 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:10.258695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754dec8-bb60-4bdc-bb28-fcea45aecbc4 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:10.271362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9195a93-f892-4aa3-90ec-f21aec4820ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:10.272177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9195a93-f892-4aa3-90ec-f21aec4820ea admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:10.272458 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9195a93-f892-4aa3-90ec-f21aec4820ea admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:10.272734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9195a93-f892-4aa3-90ec-f21aec4820ea admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:10.273620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9195a93-f892-4aa3-90ec-f21aec4820ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:10.275321 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9195a93-f892-4aa3-90ec-f21aec4820ea admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:10.286905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1369/2746] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:10 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:10.290869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9195a93-f892-4aa3-90ec-f21aec4820ea admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:10.302457 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9195a93-f892-4aa3-90ec-f21aec4820ea admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:10.307661 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17052d1-f35e-46e4-a6db-ae178a134b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:10.309813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17052d1-f35e-46e4-a6db-ae178a134b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:10.309813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17052d1-f35e-46e4-a6db-ae178a134b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:10.309813 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17052d1-f35e-46e4-a6db-ae178a134b2b admin admin] PATH_INFO: `/v3/projects/fc5c5f87b596432db9fad9ac0f8ee97d/users/7fcf54bdbf2c49fc8c890f19c3c9e6d6/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:10.325630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17052d1-f35e-46e4-a6db-ae178a134b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5c5f87b596432db9fad9ac0f8ee97d, user_id=7fcf54bdbf2c49fc8c890f19c3c9e6d6, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:10.328511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17052d1-f35e-46e4-a6db-ae178a134b2b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:10.457090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1370/2747] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:10 2022] PUT /identity/v3/projects/fc5c5f87b596432db9fad9ac0f8ee97d/users/7fcf54bdbf2c49fc8c890f19c3c9e6d6/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:12:10.469196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b86f3-74a1-43ce-8d2f-249cf4b897f0 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:10.469678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b86f3-74a1-43ce-8d2f-249cf4b897f0 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:10.470087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b86f3-74a1-43ce-8d2f-249cf4b897f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:10.474140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:10.481315 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '346b16eccc4e4535b3482e22e115249d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:10.481616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '346b16eccc4e4535b3482e22e115249d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:10.482236 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '346b16eccc4e4535b3482e22e115249d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:10.501822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-ba4b86f3-74a1-43ce-8d2f-249cf4b897f0 None None] MFA Rules not processed for user `7fcf54bdbf2c49fc8c890f19c3c9e6d6`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:10.503253 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ba4b86f3-74a1-43ce-8d2f-249cf4b897f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:10.578875 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1371/2748] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:10.583468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b86f3-74a1-43ce-8d2f-249cf4b897f0 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:10.584747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ba4b86f3-74a1-43ce-8d2f-249cf4b897f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:10.637214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:10.646064 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '346b16eccc4e4535b3482e22e115249d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:10.656500 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b86f3-74a1-43ce-8d2f-249cf4b897f0 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:10.659743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-0afdc754-217e-411d-bf41-3e291b8ef957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '346b16eccc4e4535b3482e22e115249d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:10.668799 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1c8d2-5335-4985-b819-b512f1e56596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:10.669418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1c8d2-5335-4985-b819-b512f1e56596 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:10.669693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1c8d2-5335-4985-b819-b512f1e56596 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:10.670000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1c8d2-5335-4985-b819-b512f1e56596 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:10.674432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1c8d2-5335-4985-b819-b512f1e56596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:10.674432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1c8d2-5335-4985-b819-b512f1e56596 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:10.674737 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1378/2749] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:10 2022] DELETE /identity/v3/users/346b16eccc4e4535b3482e22e115249d => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:12:10.677972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba5ea-029f-4325-bec1-40b92f4f248f None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:10.678621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba5ea-029f-4325-bec1-40b92f4f248f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:10.679163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaba5ea-029f-4325-bec1-40b92f4f248f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:10.699628 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-bfaba5ea-029f-4325-bec1-40b92f4f248f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:10.700852 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-bfaba5ea-029f-4325-bec1-40b92f4f248f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:10.763356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1379/2750] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:10.774170 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaba5ea-029f-4325-bec1-40b92f4f248f None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:10.775621 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-bfaba5ea-029f-4325-bec1-40b92f4f248f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:10.820774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1372/2751] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:10 2022] POST /identity/v3/projects => generated 364 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:12:10.824936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1c8d2-5335-4985-b819-b512f1e56596 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:10.848966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaba5ea-029f-4325-bec1-40b92f4f248f None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:10.903674 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c331cd-1e8e-434f-add7-ac69b6a35099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:10.904046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1c8d2-5335-4985-b819-b512f1e56596 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:10.904287 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c331cd-1e8e-434f-add7-ac69b6a35099 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:10.904561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c331cd-1e8e-434f-add7-ac69b6a35099 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:10.904830 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c331cd-1e8e-434f-add7-ac69b6a35099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:10.910969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc46822-97f0-4ecc-b098-59b00865dbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:10.911625 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46822-97f0-4ecc-b098-59b00865dbef admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:10.916605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46822-97f0-4ecc-b098-59b00865dbef admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:10.916605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46822-97f0-4ecc-b098-59b00865dbef admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:10.916605 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc46822-97f0-4ecc-b098-59b00865dbef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:10.916876 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c331cd-1e8e-434f-add7-ac69b6a35099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:10.916980 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc46822-97f0-4ecc-b098-59b00865dbef admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:10.917073 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c331cd-1e8e-434f-add7-ac69b6a35099 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:10.926162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1380/2752] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:10 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:12:11.063890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1373/2753] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:10 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:12:11.067123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c331cd-1e8e-434f-add7-ac69b6a35099 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:11.077495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c331cd-1e8e-434f-add7-ac69b6a35099 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:11.082281 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828899b-d21d-4555-81bd-c741934540fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:11.082884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828899b-d21d-4555-81bd-c741934540fd admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:11.083209 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828899b-d21d-4555-81bd-c741934540fd admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:11.083485 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828899b-d21d-4555-81bd-c741934540fd admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:11.084122 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828899b-d21d-4555-81bd-c741934540fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:11.085719 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828899b-d21d-4555-81bd-c741934540fd admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:11.097504 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1381/2754] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:11 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:11.103607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc46822-97f0-4ecc-b098-59b00865dbef admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:11.115444 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc46822-97f0-4ecc-b098-59b00865dbef admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:11.120692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c407b32-b7f2-43f7-a870-9a9ffa2a64c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:11.121407 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c407b32-b7f2-43f7-a870-9a9ffa2a64c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:11.121682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c407b32-b7f2-43f7-a870-9a9ffa2a64c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:11.122008 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c407b32-b7f2-43f7-a870-9a9ffa2a64c8 admin admin] PATH_INFO: `/v3/projects/5445387fc1ac41efaff10908718237c9/users/29aef048a45c470f919763c771560a6f/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:11.141554 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c407b32-b7f2-43f7-a870-9a9ffa2a64c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5445387fc1ac41efaff10908718237c9, user_id=29aef048a45c470f919763c771560a6f, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:11.145115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c407b32-b7f2-43f7-a870-9a9ffa2a64c8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:11.561511 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1374/2755] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:11 2022] PUT /identity/v3/projects/5445387fc1ac41efaff10908718237c9/users/29aef048a45c470f919763c771560a6f/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:11.566690 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828899b-d21d-4555-81bd-c741934540fd admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:11.573298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:11.577375 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828899b-d21d-4555-81bd-c741934540fd admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:11.592835 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c407b32-b7f2-43f7-a870-9a9ffa2a64c8 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:11.595338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-8c407b32-b7f2-43f7-a870-9a9ffa2a64c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:11.649673 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b5205-38fe-463b-b36d-5b2e5be653e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:11.650294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b5205-38fe-463b-b36d-5b2e5be653e2 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:11.650574 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b5205-38fe-463b-b36d-5b2e5be653e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:11.650843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b5205-38fe-463b-b36d-5b2e5be653e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:11.651578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b5205-38fe-463b-b36d-5b2e5be653e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:11.653161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b5205-38fe-463b-b36d-5b2e5be653e2 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:11.666914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1382/2756] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:11 2022] GET /identity/v3/roles => generated 1427 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:11.671559 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c407b32-b7f2-43f7-a870-9a9ffa2a64c8 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:11.672825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b5205-38fe-463b-b36d-5b2e5be653e2 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:11.676174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:11.676974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:11.677256 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:11.677525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] PATH_INFO: `/v3/projects/58d483bf70a94a4b99b987c159e865df` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:11.678903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d483bf70a94a4b99b987c159e865df)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:11.680838 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:11.683542 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b5205-38fe-463b-b36d-5b2e5be653e2 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:11.690912 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785751c-98df-4220-ba63-874266df781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:11.691578 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785751c-98df-4220-ba63-874266df781b admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:11.691872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785751c-98df-4220-ba63-874266df781b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:11.692146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785751c-98df-4220-ba63-874266df781b admin admin] PATH_INFO: `/v3/projects/5445387fc1ac41efaff10908718237c9/users/29aef048a45c470f919763c771560a6f/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:11.694332 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785751c-98df-4220-ba63-874266df781b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5445387fc1ac41efaff10908718237c9, user_id=29aef048a45c470f919763c771560a6f, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:11.697349 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785751c-98df-4220-ba63-874266df781b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:11.711955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1383/2757] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:11 2022] PUT /identity/v3/projects/5445387fc1ac41efaff10908718237c9/users/29aef048a45c470f919763c771560a6f/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:11.724618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20525012-8290-434d-bc7e-19440cac1dc9 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:11.724914 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20525012-8290-434d-bc7e-19440cac1dc9 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:11.725188 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20525012-8290-434d-bc7e-19440cac1dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:11.754897 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-20525012-8290-434d-bc7e-19440cac1dc9 None None] MFA Rules not processed for user `29aef048a45c470f919763c771560a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:11.761401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-20525012-8290-434d-bc7e-19440cac1dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:11.832033 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1384/2758] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:11.843926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:11.846779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20525012-8290-434d-bc7e-19440cac1dc9 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:11.848439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-20525012-8290-434d-bc7e-19440cac1dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:11.852783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d483bf70a94a4b99b987c159e865df', 'request_id': 'req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:11.863614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d483bf70a94a4b99b987c159e865df', 'request_id': 'req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:11.916127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20525012-8290-434d-bc7e-19440cac1dc9 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:11.969125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5636bd-4372-4bf5-af0d-3ede7f22eadc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:11.969731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5636bd-4372-4bf5-af0d-3ede7f22eadc service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:11.969941 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d483bf70a94a4b99b987c159e865df', 'request_id': 'req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:11.970137 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5636bd-4372-4bf5-af0d-3ede7f22eadc service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:11.970304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5636bd-4372-4bf5-af0d-3ede7f22eadc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:11.981940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] The token cache is being invalidate because project 58d483bf70a94a4b99b987c159e865df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:11.983089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:11.984281 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:11.986054 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1375/2759] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:11 2022] DELETE /identity/v3/projects/58d483bf70a94a4b99b987c159e865df => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:11.986218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:11.991055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:11.993207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:12.041793 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5636bd-4372-4bf5-af0d-3ede7f22eadc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:12.043277 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5636bd-4372-4bf5-af0d-3ede7f22eadc service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:12.054120 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1385/2760] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:12:12.068272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed7636-97a0-4fb1-bdb9-57ae8cfb55a2 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:12.075088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:12.075778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:12.076085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:12.076431 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] PATH_INFO: `/v3/users/08cc0465a9d34292b048b0ba31ccb690` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:12.077757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] RBAC: Authorizing `identity:delete_user(user_id=08cc0465a9d34292b048b0ba31ccb690)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:12.079856 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:12.289764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5636bd-4372-4bf5-af0d-3ede7f22eadc service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:12.300003 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5636bd-4372-4bf5-af0d-3ede7f22eadc service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:12.304653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89980c-526a-474d-a058-5675e6379851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:12.308340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89980c-526a-474d-a058-5675e6379851 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:12.308340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89980c-526a-474d-a058-5675e6379851 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:12.308340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89980c-526a-474d-a058-5675e6379851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:12.308340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8e89980c-526a-474d-a058-5675e6379851 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:12.313608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08cc0465a9d34292b048b0ba31ccb690'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:12.314323 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08cc0465a9d34292b048b0ba31ccb690'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:12.314930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08cc0465a9d34292b048b0ba31ccb690'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:12.369018 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89980c-526a-474d-a058-5675e6379851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:12.370463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89980c-526a-474d-a058-5675e6379851 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:12.380442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1386/2761] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:12:12.457482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08cc0465a9d34292b048b0ba31ccb690'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:12.470770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08cc0465a9d34292b048b0ba31ccb690'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:12.485234 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1376/2762] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:11 2022] DELETE /identity/v3/users/08cc0465a9d34292b048b0ba31ccb690 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:12.943687 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89980c-526a-474d-a058-5675e6379851 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:12.956138 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89980c-526a-474d-a058-5675e6379851 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:13.017357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:13.018005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:13.018286 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:13.018560 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] PATH_INFO: `/v3/projects/b1d620fff8b340ad9f79d511fbbb5336` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:13.019731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d620fff8b340ad9f79d511fbbb5336)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:13.025615 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:13.046072 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:13.189292 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d620fff8b340ad9f79d511fbbb5336', 'request_id': 'req-8fae7558-dac4-4d20-ab03-cf8bcede488a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:13.196140 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d620fff8b340ad9f79d511fbbb5336', 'request_id': 'req-8fae7558-dac4-4d20-ab03-cf8bcede488a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:13.202576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d620fff8b340ad9f79d511fbbb5336', 'request_id': 'req-8fae7558-dac4-4d20-ab03-cf8bcede488a'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:13.326051 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] The token cache is being invalidate because project b1d620fff8b340ad9f79d511fbbb5336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:13.327171 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:13.327679 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:13.329275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1387/2763] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:12 2022] DELETE /identity/v3/projects/b1d620fff8b340ad9f79d511fbbb5336 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:13.333455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:13.334970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:13.350157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bd56a5-b951-42b5-a564-a144759d125e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:13.397859 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:13.410929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:13.411614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:13.411919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:13.412184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] PATH_INFO: `/v3/users/ff4b9f161ff64a51b7e32f7f22c30289` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:13.413309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff4b9f161ff64a51b7e32f7f22c30289)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:13.415254 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:13.586779 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff4b9f161ff64a51b7e32f7f22c30289'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:13.587495 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff4b9f161ff64a51b7e32f7f22c30289'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:13.588113 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff4b9f161ff64a51b7e32f7f22c30289'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:13.692787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff4b9f161ff64a51b7e32f7f22c30289'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:13.706181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff4b9f161ff64a51b7e32f7f22c30289'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:13.718679 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1377/2764] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:13 2022] DELETE /identity/v3/users/ff4b9f161ff64a51b7e32f7f22c30289 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:14.353950 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:14.359065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:14.373128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae7558-dac4-4d20-ab03-cf8bcede488a admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:14.449500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:14.450151 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:14.450426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:14.450699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] PATH_INFO: `/v3/projects/f4261c49b84d4a328451783fe1fd48d3` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:14.452016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4261c49b84d4a328451783fe1fd48d3)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:14.453996 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:14.612396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4261c49b84d4a328451783fe1fd48d3', 'request_id': 'req-09c85610-0add-47b6-b612-ca8ddef345ae'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:14.620561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4261c49b84d4a328451783fe1fd48d3', 'request_id': 'req-09c85610-0add-47b6-b612-ca8ddef345ae'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:14.627604 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4261c49b84d4a328451783fe1fd48d3', 'request_id': 'req-09c85610-0add-47b6-b612-ca8ddef345ae'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:14.736519 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] The token cache is being invalidate because project f4261c49b84d4a328451783fe1fd48d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:14.742289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:14.742289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-09c85610-0add-47b6-b612-ca8ddef345ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:14.742289 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1388/2765] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:14 2022] DELETE /identity/v3/projects/f4261c49b84d4a328451783fe1fd48d3 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:16.307833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:16.309294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:16.325064 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56a674e-1d13-4422-82bb-d1b6e2e10490 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:16.392022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:16.392656 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:16.392929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:16.393191 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] PATH_INFO: `/v3/users/7fcf54bdbf2c49fc8c890f19c3c9e6d6` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:16.394325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fcf54bdbf2c49fc8c890f19c3c9e6d6)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:16.396247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:16.689276 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fcf54bdbf2c49fc8c890f19c3c9e6d6'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:16.689937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fcf54bdbf2c49fc8c890f19c3c9e6d6'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:16.690541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fcf54bdbf2c49fc8c890f19c3c9e6d6'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:16.787347 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fcf54bdbf2c49fc8c890f19c3c9e6d6'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:16.801214 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:16.802569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fcf54bdbf2c49fc8c890f19c3c9e6d6'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:16.815510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1378/2766] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:16 2022] DELETE /identity/v3/users/7fcf54bdbf2c49fc8c890f19c3c9e6d6 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:16.820280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299606-ee24-40bf-bbdf-bae7127c2664 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:16.820783 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299606-ee24-40bf-bbdf-bae7127c2664 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:16.820903 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7299606-ee24-40bf-bbdf-bae7127c2664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:16.841559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c7299606-ee24-40bf-bbdf-bae7127c2664 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:16.843567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c7299606-ee24-40bf-bbdf-bae7127c2664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:16.877191 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:16.903419 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1389/2767] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:16.914259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:16.915763 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:16.976810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa156f5-6ccf-4c09-bbf0-69b2b8639a0f admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:16.980567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69ecac-35a7-4c9e-a8ef-4a9c909fab8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:16.981200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69ecac-35a7-4c9e-a8ef-4a9c909fab8d service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:16.981468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69ecac-35a7-4c9e-a8ef-4a9c909fab8d service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:16.981730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69ecac-35a7-4c9e-a8ef-4a9c909fab8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:16.993263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69ecac-35a7-4c9e-a8ef-4a9c909fab8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:17.006274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69ecac-35a7-4c9e-a8ef-4a9c909fab8d service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:17.006274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1379/2768] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:17.491899 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299606-ee24-40bf-bbdf-bae7127c2664 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:17.494012 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c7299606-ee24-40bf-bbdf-bae7127c2664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:17.510308 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7299606-ee24-40bf-bbdf-bae7127c2664 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:17.516828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:17.517868 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:17.518226 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:17.518576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] PATH_INFO: `/v3/projects/fc5c5f87b596432db9fad9ac0f8ee97d` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:17.519858 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc5c5f87b596432db9fad9ac0f8ee97d)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:17.522790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:17.667187 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc5c5f87b596432db9fad9ac0f8ee97d', 'request_id': 'req-dd619361-e5b0-4a01-aeed-9d496b07255d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:17.670872 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc5c5f87b596432db9fad9ac0f8ee97d', 'request_id': 'req-dd619361-e5b0-4a01-aeed-9d496b07255d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:17.677013 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc5c5f87b596432db9fad9ac0f8ee97d', 'request_id': 'req-dd619361-e5b0-4a01-aeed-9d496b07255d'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:17.783878 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] The token cache is being invalidate because project fc5c5f87b596432db9fad9ac0f8ee97d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:17.784241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:17.784994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:17.786570 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1390/2769] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:17 2022] DELETE /identity/v3/projects/fc5c5f87b596432db9fad9ac0f8ee97d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:17.790644 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69ecac-35a7-4c9e-a8ef-4a9c909fab8d service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:17.802715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69ecac-35a7-4c9e-a8ef-4a9c909fab8d service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:17.879734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:17.880667 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:17.880938 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:17.881204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] PATH_INFO: `/v3/users/29aef048a45c470f919763c771560a6f` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:17.882547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29aef048a45c470f919763c771560a6f)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:17.884490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:18.106541 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29aef048a45c470f919763c771560a6f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:18.107239 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29aef048a45c470f919763c771560a6f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:18.108529 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29aef048a45c470f919763c771560a6f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:18.189670 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29aef048a45c470f919763c771560a6f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:18.203746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a460859f-cb28-46f2-bfe8-fc87309db3e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29aef048a45c470f919763c771560a6f'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:18.216354 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1380/2770] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:17 2022] DELETE /identity/v3/users/29aef048a45c470f919763c771560a6f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:18.733356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:18.735007 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:18.756342 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd619361-e5b0-4a01-aeed-9d496b07255d admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:18.821179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:18.821853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:18.822135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:18.822411 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] PATH_INFO: `/v3/projects/5445387fc1ac41efaff10908718237c9` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:18.823622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] RBAC: Authorizing `identity:delete_project(project_id=5445387fc1ac41efaff10908718237c9)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:18.825711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:18.965113 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5445387fc1ac41efaff10908718237c9', 'request_id': 'req-bc3b8afd-7671-414d-a990-fcd024173dae'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:18.971381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5445387fc1ac41efaff10908718237c9', 'request_id': 'req-bc3b8afd-7671-414d-a990-fcd024173dae'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:18.977071 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5445387fc1ac41efaff10908718237c9', 'request_id': 'req-bc3b8afd-7671-414d-a990-fcd024173dae'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:18.978731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:19.069303 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] The token cache is being invalidate because project 5445387fc1ac41efaff10908718237c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:19.069590 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:19.070139 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-bc3b8afd-7671-414d-a990-fcd024173dae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:19.071702 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1391/2771] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:18 2022] DELETE /identity/v3/projects/5445387fc1ac41efaff10908718237c9 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:19.076830 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18701948-257f-48a1-94a9-34fe76573add None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:19.077587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18701948-257f-48a1-94a9-34fe76573add None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:19.077587 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18701948-257f-48a1-94a9-34fe76573add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:19.097252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-18701948-257f-48a1-94a9-34fe76573add None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:19.098465 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-18701948-257f-48a1-94a9-34fe76573add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:19.102957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:19.162198 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1381/2772] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:19.171143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa4117-f4e9-457b-8a40-2c7bc7064ad9 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:19.171436 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa4117-f4e9-457b-8a40-2c7bc7064ad9 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:19.171714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa4117-f4e9-457b-8a40-2c7bc7064ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:19.193569 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-b4fa4117-f4e9-457b-8a40-2c7bc7064ad9 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:19.195495 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-b4fa4117-f4e9-457b-8a40-2c7bc7064ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:19.203501 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1392/2773] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:19.207860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18701948-257f-48a1-94a9-34fe76573add None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:19.218991 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18701948-257f-48a1-94a9-34fe76573add None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:19.223637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94383e-3858-414f-91d0-718d2aa17a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:19.224243 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94383e-3858-414f-91d0-718d2aa17a5b admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:19.230085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94383e-3858-414f-91d0-718d2aa17a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:19.230085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94383e-3858-414f-91d0-718d2aa17a5b admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:19.230085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94383e-3858-414f-91d0-718d2aa17a5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:12:19.230085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94383e-3858-414f-91d0-718d2aa17a5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:19.230085 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94383e-3858-414f-91d0-718d2aa17a5b admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:19.241105 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1382/2774] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:12:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:12:19.245750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa4117-f4e9-457b-8a40-2c7bc7064ad9 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:19.256279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa4117-f4e9-457b-8a40-2c7bc7064ad9 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:19.261178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35225ec-f282-4948-8899-fbc7f2bdcda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:19.261779 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35225ec-f282-4948-8899-fbc7f2bdcda7 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:19.264265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35225ec-f282-4948-8899-fbc7f2bdcda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:19.264265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35225ec-f282-4948-8899-fbc7f2bdcda7 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:19.264265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35225ec-f282-4948-8899-fbc7f2bdcda7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:19.265096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35225ec-f282-4948-8899-fbc7f2bdcda7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:19.390246 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1393/2775] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:19 2022] POST /identity/v3/projects => generated 394 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:19.394901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94383e-3858-414f-91d0-718d2aa17a5b admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:19.406709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94383e-3858-414f-91d0-718d2aa17a5b admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:19.473061 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280c5d1-3efa-410c-bdc2-b518c6e893ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:19.473593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280c5d1-3efa-410c-bdc2-b518c6e893ec admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:19.473898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280c5d1-3efa-410c-bdc2-b518c6e893ec admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:19.474167 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280c5d1-3efa-410c-bdc2-b518c6e893ec admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:19.474792 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280c5d1-3efa-410c-bdc2-b518c6e893ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:19.476818 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280c5d1-3efa-410c-bdc2-b518c6e893ec admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:19.565723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1383/2776] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:19 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:19.569926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35225ec-f282-4948-8899-fbc7f2bdcda7 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:19.583288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35225ec-f282-4948-8899-fbc7f2bdcda7 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:19.588167 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b5ebf-c8c4-4915-815e-38598ac1656f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:19.588773 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b5ebf-c8c4-4915-815e-38598ac1656f admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:19.589049 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b5ebf-c8c4-4915-815e-38598ac1656f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:19.589330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b5ebf-c8c4-4915-815e-38598ac1656f admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:19.590009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b5ebf-c8c4-4915-815e-38598ac1656f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:19.591731 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13b5ebf-c8c4-4915-815e-38598ac1656f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:19.602239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1394/2777] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:19 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:19.606390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280c5d1-3efa-410c-bdc2-b518c6e893ec admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:19.616810 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280c5d1-3efa-410c-bdc2-b518c6e893ec admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:19.621650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381daed-9ba9-4482-b649-747c45a3522f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:19.622248 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381daed-9ba9-4482-b649-747c45a3522f admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:19.622526 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381daed-9ba9-4482-b649-747c45a3522f admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:19.622796 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381daed-9ba9-4482-b649-747c45a3522f admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:19.623459 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381daed-9ba9-4482-b649-747c45a3522f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:19.625727 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381daed-9ba9-4482-b649-747c45a3522f admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:19.637285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1384/2778] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:19 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:19.641840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b5ebf-c8c4-4915-815e-38598ac1656f admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:19.652339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b5ebf-c8c4-4915-815e-38598ac1656f admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:19.658308 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac8785-1f7c-441a-ac94-b1b5c9b65fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:19.658957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac8785-1f7c-441a-ac94-b1b5c9b65fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:19.660508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac8785-1f7c-441a-ac94-b1b5c9b65fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:19.660508 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac8785-1f7c-441a-ac94-b1b5c9b65fdb admin admin] PATH_INFO: `/v3/projects/f302d9d5eb6d4caab63448bd53c1eb8b/users/8398c0726b8d4537aaa00754462838fe/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:19.676785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac8785-1f7c-441a-ac94-b1b5c9b65fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f302d9d5eb6d4caab63448bd53c1eb8b, user_id=8398c0726b8d4537aaa00754462838fe, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:19.681861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac8785-1f7c-441a-ac94-b1b5c9b65fdb admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:19.781986 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1395/2779] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:19 2022] PUT /identity/v3/projects/f302d9d5eb6d4caab63448bd53c1eb8b/users/8398c0726b8d4537aaa00754462838fe/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:19.792593 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf988c-5cb6-4807-bad8-f52eaf108b58 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:19.792858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf988c-5cb6-4807-bad8-f52eaf108b58 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:19.793129 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf988c-5cb6-4807-bad8-f52eaf108b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:19.822361 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-15cf988c-5cb6-4807-bad8-f52eaf108b58 None None] MFA Rules not processed for user `8398c0726b8d4537aaa00754462838fe`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:19.823640 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-15cf988c-5cb6-4807-bad8-f52eaf108b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:19.901010 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1385/2780] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:19 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:19.905882 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac8785-1f7c-441a-ac94-b1b5c9b65fdb admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:19.976224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac8785-1f7c-441a-ac94-b1b5c9b65fdb admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:19.981162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efe73a-c3b9-42ed-9837-adc563084998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:19.981807 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efe73a-c3b9-42ed-9837-adc563084998 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:19.982062 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efe73a-c3b9-42ed-9837-adc563084998 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:19.982334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efe73a-c3b9-42ed-9837-adc563084998 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:19.982979 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efe73a-c3b9-42ed-9837-adc563084998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:19.984928 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efe73a-c3b9-42ed-9837-adc563084998 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:20.066466 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1396/2781] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:19 2022] POST /identity/v3/projects => generated 392 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:20.079225 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf988c-5cb6-4807-bad8-f52eaf108b58 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:20.086203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:20.090616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf988c-5cb6-4807-bad8-f52eaf108b58 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:20.151970 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed6b7e-4b1d-4dff-89ea-2503e07d31fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:20.152586 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed6b7e-4b1d-4dff-89ea-2503e07d31fa admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:20.152920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed6b7e-4b1d-4dff-89ea-2503e07d31fa admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:20.153189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed6b7e-4b1d-4dff-89ea-2503e07d31fa admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:20.153814 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed6b7e-4b1d-4dff-89ea-2503e07d31fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:20.155893 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed6b7e-4b1d-4dff-89ea-2503e07d31fa admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:20.257285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1386/2782] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:20 2022] POST /identity/v3/users => generated 303 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:20.261811 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efe73a-c3b9-42ed-9837-adc563084998 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:20.273403 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efe73a-c3b9-42ed-9837-adc563084998 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:20.278100 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc6a08-8fbb-40c1-bb96-b0497f590c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:20.278742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc6a08-8fbb-40c1-bb96-b0497f590c2c admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:20.279037 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc6a08-8fbb-40c1-bb96-b0497f590c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:20.279368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc6a08-8fbb-40c1-bb96-b0497f590c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:20.280198 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc6a08-8fbb-40c1-bb96-b0497f590c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:20.281918 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc6a08-8fbb-40c1-bb96-b0497f590c2c admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:20.293079 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1397/2783] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:20 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:20.298499 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed6b7e-4b1d-4dff-89ea-2503e07d31fa admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:20.309957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed6b7e-4b1d-4dff-89ea-2503e07d31fa admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:20.315724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53251ad5-2462-4fb9-ba4c-9dc5150f41ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:20.316392 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53251ad5-2462-4fb9-ba4c-9dc5150f41ea admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:20.316668 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53251ad5-2462-4fb9-ba4c-9dc5150f41ea admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:20.316940 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53251ad5-2462-4fb9-ba4c-9dc5150f41ea admin admin] PATH_INFO: `/v3/projects/15570c81bf59457d8652f4f7d396159a/users/db9c4283c7c641988f44f5a0f9445a1b/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:20.333283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53251ad5-2462-4fb9-ba4c-9dc5150f41ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=15570c81bf59457d8652f4f7d396159a, user_id=db9c4283c7c641988f44f5a0f9445a1b, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:20.336498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53251ad5-2462-4fb9-ba4c-9dc5150f41ea admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:20.450869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1387/2784] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:20 2022] PUT /identity/v3/projects/15570c81bf59457d8652f4f7d396159a/users/db9c4283c7c641988f44f5a0f9445a1b/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:20.455153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc6a08-8fbb-40c1-bb96-b0497f590c2c admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:20.465564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc6a08-8fbb-40c1-bb96-b0497f590c2c admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:20.526659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4702f28-2292-4971-9aa6-84b08eafe982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:20.527311 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4702f28-2292-4971-9aa6-84b08eafe982 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:20.527591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4702f28-2292-4971-9aa6-84b08eafe982 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:20.527861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4702f28-2292-4971-9aa6-84b08eafe982 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:20.528506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4702f28-2292-4971-9aa6-84b08eafe982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:20.530724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4702f28-2292-4971-9aa6-84b08eafe982 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:20.533005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:20.541338 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1398/2785] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:20 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:20.552670 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53251ad5-2462-4fb9-ba4c-9dc5150f41ea admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:20.557663 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53251ad5-2462-4fb9-ba4c-9dc5150f41ea admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:20.562994 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b9853-9ebe-442d-925e-2558248043c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:20.563665 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b9853-9ebe-442d-925e-2558248043c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:20.563938 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b9853-9ebe-442d-925e-2558248043c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:20.564199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b9853-9ebe-442d-925e-2558248043c1 admin admin] PATH_INFO: `/v3/projects/15570c81bf59457d8652f4f7d396159a/users/db9c4283c7c641988f44f5a0f9445a1b/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:20.566272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b9853-9ebe-442d-925e-2558248043c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15570c81bf59457d8652f4f7d396159a, user_id=db9c4283c7c641988f44f5a0f9445a1b, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:20.568975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b9853-9ebe-442d-925e-2558248043c1 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:20.580933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1388/2786] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:20 2022] PUT /identity/v3/projects/15570c81bf59457d8652f4f7d396159a/users/db9c4283c7c641988f44f5a0f9445a1b/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:20.592777 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01d8ac-3f84-4bf9-924a-b90a3246cb7f None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:20.593065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01d8ac-3f84-4bf9-924a-b90a3246cb7f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:20.593376 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01d8ac-3f84-4bf9-924a-b90a3246cb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:20.628280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-2a01d8ac-3f84-4bf9-924a-b90a3246cb7f None None] MFA Rules not processed for user `db9c4283c7c641988f44f5a0f9445a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:20.629525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2a01d8ac-3f84-4bf9-924a-b90a3246cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:20.701955 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1399/2787] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:20 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:20.713357 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b9853-9ebe-442d-925e-2558248043c1 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:20.715093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-618b9853-9ebe-442d-925e-2558248043c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:20.775650 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b9853-9ebe-442d-925e-2558248043c1 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:20.779240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303ba49-505c-489b-8c9d-80cdfdd4f7a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:20.780490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303ba49-505c-489b-8c9d-80cdfdd4f7a3 service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:20.780881 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303ba49-505c-489b-8c9d-80cdfdd4f7a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:20.781148 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303ba49-505c-489b-8c9d-80cdfdd4f7a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:20.790723 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303ba49-505c-489b-8c9d-80cdfdd4f7a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:20.792168 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d303ba49-505c-489b-8c9d-80cdfdd4f7a3 service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:20.802890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1389/2788] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:20 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:20.851228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01d8ac-3f84-4bf9-924a-b90a3246cb7f None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:20.852660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-2a01d8ac-3f84-4bf9-924a-b90a3246cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:20.865261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01d8ac-3f84-4bf9-924a-b90a3246cb7f None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:20.869025 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a124c7-ed1e-4051-b562-6462e192a370 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:20.869629 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a124c7-ed1e-4051-b562-6462e192a370 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:20.869905 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a124c7-ed1e-4051-b562-6462e192a370 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:20.870228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a124c7-ed1e-4051-b562-6462e192a370 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:20.930676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a124c7-ed1e-4051-b562-6462e192a370 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:20.932134 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a124c7-ed1e-4051-b562-6462e192a370 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:20.941785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1400/2789] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:20 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:30.391946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303ba49-505c-489b-8c9d-80cdfdd4f7a3 service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:30.458493 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303ba49-505c-489b-8c9d-80cdfdd4f7a3 service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:30.463566 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:30.468012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:30.468012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:30.468012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] PATH_INFO: `/v3/users/8398c0726b8d4537aaa00754462838fe` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:30.468012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8398c0726b8d4537aaa00754462838fe)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:30.468012 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:30.469555 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:30.714088 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8398c0726b8d4537aaa00754462838fe'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:30.714737 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8398c0726b8d4537aaa00754462838fe'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:30.715420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8398c0726b8d4537aaa00754462838fe'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:30.787911 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8398c0726b8d4537aaa00754462838fe'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:30.806404 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8398c0726b8d4537aaa00754462838fe'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:30.830172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1390/2790] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:30 2022] DELETE /identity/v3/users/8398c0726b8d4537aaa00754462838fe => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:12:30.835506 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85b5ed-36d3-4aac-8572-4bc80180037a None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:30.836006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85b5ed-36d3-4aac-8572-4bc80180037a None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:30.836164 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85b5ed-36d3-4aac-8572-4bc80180037a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:30.858710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-3e85b5ed-36d3-4aac-8572-4bc80180037a None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:30.858710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3e85b5ed-36d3-4aac-8572-4bc80180037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:30.923841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1401/2791] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:30.937406 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:30.938951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:31.001324 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8936db-50b3-40ad-9bf4-e378d6adafd8 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:31.005019 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d3e36-266e-4076-8d76-1791675c5475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:31.005624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d3e36-266e-4076-8d76-1791675c5475 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:31.005976 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d3e36-266e-4076-8d76-1791675c5475 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:31.006165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d3e36-266e-4076-8d76-1791675c5475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:31.020883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d3e36-266e-4076-8d76-1791675c5475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:31.020883 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d3e36-266e-4076-8d76-1791675c5475 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:31.029455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1391/2792] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:31.527487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85b5ed-36d3-4aac-8572-4bc80180037a None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:31.528989 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3e85b5ed-36d3-4aac-8572-4bc80180037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:31.541834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85b5ed-36d3-4aac-8572-4bc80180037a None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:31.547382 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:31.548114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:31.548398 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:31.548685 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] PATH_INFO: `/v3/projects/f302d9d5eb6d4caab63448bd53c1eb8b` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:31.550067 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f302d9d5eb6d4caab63448bd53c1eb8b)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:31.552060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:31.572123 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:31.696245 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f302d9d5eb6d4caab63448bd53c1eb8b', 'request_id': 'req-946d759b-c148-4c6f-ac8b-729660bb4ba8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:31.704661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f302d9d5eb6d4caab63448bd53c1eb8b', 'request_id': 'req-946d759b-c148-4c6f-ac8b-729660bb4ba8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:31.711565 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f302d9d5eb6d4caab63448bd53c1eb8b', 'request_id': 'req-946d759b-c148-4c6f-ac8b-729660bb4ba8'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:31.817280 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] The token cache is being invalidate because project f302d9d5eb6d4caab63448bd53c1eb8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:31.817568 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:31.818239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:31.819948 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1402/2793] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:31 2022] DELETE /identity/v3/projects/f302d9d5eb6d4caab63448bd53c1eb8b => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:31.824274 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d3e36-266e-4076-8d76-1791675c5475 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:31.835172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d3e36-266e-4076-8d76-1791675c5475 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:31.896571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:31.897224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:31.897501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:31.897771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] PATH_INFO: `/v3/users/db9c4283c7c641988f44f5a0f9445a1b` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:31.898937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9c4283c7c641988f44f5a0f9445a1b)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:31.900901 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:31.926560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:32.082005 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9c4283c7c641988f44f5a0f9445a1b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:32.082694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9c4283c7c641988f44f5a0f9445a1b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:32.083333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9c4283c7c641988f44f5a0f9445a1b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:32.152155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9c4283c7c641988f44f5a0f9445a1b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:32.162391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-278a239b-63ba-4c0a-8e23-9ba543ed8ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9c4283c7c641988f44f5a0f9445a1b'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:32.175006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1392/2794] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:31 2022] DELETE /identity/v3/users/db9c4283c7c641988f44f5a0f9445a1b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:32.620222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:32.621935 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:32.636932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d759b-c148-4c6f-ac8b-729660bb4ba8 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:32.706611 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:32.707309 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:32.707586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:32.707856 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] PATH_INFO: `/v3/projects/15570c81bf59457d8652f4f7d396159a` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:32.709092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] RBAC: Authorizing `identity:delete_project(project_id=15570c81bf59457d8652f4f7d396159a)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:32.711045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:32.824395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15570c81bf59457d8652f4f7d396159a', 'request_id': 'req-9d78652b-f83e-42eb-837c-c41e0c02a79b'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:32.831786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15570c81bf59457d8652f4f7d396159a', 'request_id': 'req-9d78652b-f83e-42eb-837c-c41e0c02a79b'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:32.837627 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15570c81bf59457d8652f4f7d396159a', 'request_id': 'req-9d78652b-f83e-42eb-837c-c41e0c02a79b'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:32.914128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] The token cache is being invalidate because project 15570c81bf59457d8652f4f7d396159a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:32.914416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:32.915177 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-9d78652b-f83e-42eb-837c-c41e0c02a79b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:32.916799 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1403/2795] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:32 2022] DELETE /identity/v3/projects/15570c81bf59457d8652f4f7d396159a => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:12:32.921693 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661adc5-be30-45c8-adad-1fb24e85694f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:32.922345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661adc5-be30-45c8-adad-1fb24e85694f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:32.922345 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661adc5-be30-45c8-adad-1fb24e85694f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:32.943983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-f661adc5-be30-45c8-adad-1fb24e85694f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:32.945213 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f661adc5-be30-45c8-adad-1fb24e85694f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:33.017123 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1393/2796] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:33.026000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b55b7-f2bd-4ee2-8ce4-530a8cb38589 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:33.026318 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b55b7-f2bd-4ee2-8ce4-530a8cb38589 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:33.026693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b55b7-f2bd-4ee2-8ce4-530a8cb38589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:33.046707 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-ef5b55b7-f2bd-4ee2-8ce4-530a8cb38589 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:33.047952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-ef5b55b7-f2bd-4ee2-8ce4-530a8cb38589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:33.056356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1404/2797] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:33.060682 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661adc5-be30-45c8-adad-1fb24e85694f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:33.070946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661adc5-be30-45c8-adad-1fb24e85694f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:33.075562 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c523b-4e61-491b-8e53-5e34fddecc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:33.076161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c523b-4e61-491b-8e53-5e34fddecc6d admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:33.076434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c523b-4e61-491b-8e53-5e34fddecc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:33.076702 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c523b-4e61-491b-8e53-5e34fddecc6d admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:33.077335 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c523b-4e61-491b-8e53-5e34fddecc6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:12:33.077631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c523b-4e61-491b-8e53-5e34fddecc6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:33.079222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4c523b-4e61-491b-8e53-5e34fddecc6d admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:33.091842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1394/2798] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:12:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:12:33.097074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b55b7-f2bd-4ee2-8ce4-530a8cb38589 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:33.106446 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b55b7-f2bd-4ee2-8ce4-530a8cb38589 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:33.111046 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d05ea8-936c-415f-ae6f-98abb61c2cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:33.111696 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d05ea8-936c-415f-ae6f-98abb61c2cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:33.111977 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d05ea8-936c-415f-ae6f-98abb61c2cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:33.112244 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d05ea8-936c-415f-ae6f-98abb61c2cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:33.112861 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d05ea8-936c-415f-ae6f-98abb61c2cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:33.114759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d05ea8-936c-415f-ae6f-98abb61c2cf1 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:33.231224 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1405/2799] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:33 2022] POST /identity/v3/projects => generated 410 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:33.235426 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c523b-4e61-491b-8e53-5e34fddecc6d admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:33.245997 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4c523b-4e61-491b-8e53-5e34fddecc6d admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:33.306038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79175ba3-34ef-4d30-9951-70657057a1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:33.306658 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175ba3-34ef-4d30-9951-70657057a1e3 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:33.306934 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175ba3-34ef-4d30-9951-70657057a1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:33.307232 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79175ba3-34ef-4d30-9951-70657057a1e3 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:33.307860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79175ba3-34ef-4d30-9951-70657057a1e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:33.310026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79175ba3-34ef-4d30-9951-70657057a1e3 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:33.414344 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1395/2800] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:33 2022] POST /identity/v3/users => generated 306 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:33.418667 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d05ea8-936c-415f-ae6f-98abb61c2cf1 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:33.431333 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d05ea8-936c-415f-ae6f-98abb61c2cf1 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:33.435383 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6daa5f-2de7-4e02-92ea-6f8773af74b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:33.437765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6daa5f-2de7-4e02-92ea-6f8773af74b9 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:33.437765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6daa5f-2de7-4e02-92ea-6f8773af74b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:33.437765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6daa5f-2de7-4e02-92ea-6f8773af74b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:33.437765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6daa5f-2de7-4e02-92ea-6f8773af74b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:33.438886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6daa5f-2de7-4e02-92ea-6f8773af74b9 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:33.441110 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:33.451270 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1406/2801] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:33 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:33.456570 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79175ba3-34ef-4d30-9951-70657057a1e3 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:33.467376 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79175ba3-34ef-4d30-9951-70657057a1e3 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:33.472757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79745f6-a2e3-402b-ad63-2c0919503744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:33.473362 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79745f6-a2e3-402b-ad63-2c0919503744 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:33.473643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79745f6-a2e3-402b-ad63-2c0919503744 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:33.473924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79745f6-a2e3-402b-ad63-2c0919503744 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:33.474568 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79745f6-a2e3-402b-ad63-2c0919503744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:33.476201 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79745f6-a2e3-402b-ad63-2c0919503744 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:33.488350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1396/2802] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:33 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:33.492913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6daa5f-2de7-4e02-92ea-6f8773af74b9 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:33.504774 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6daa5f-2de7-4e02-92ea-6f8773af74b9 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:33.509515 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394b7e8-5072-4fb1-a99b-5a128c80e08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:33.510178 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394b7e8-5072-4fb1-a99b-5a128c80e08d admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:33.510473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394b7e8-5072-4fb1-a99b-5a128c80e08d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:33.510723 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8394b7e8-5072-4fb1-a99b-5a128c80e08d admin admin] PATH_INFO: `/v3/projects/bff251349a5f408fbec8f389a96eee2a/users/e37a7019f2f54ce49d9509b016ac99b6/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:33.526271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394b7e8-5072-4fb1-a99b-5a128c80e08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff251349a5f408fbec8f389a96eee2a, user_id=e37a7019f2f54ce49d9509b016ac99b6, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:33.529331 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8394b7e8-5072-4fb1-a99b-5a128c80e08d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:33.593826 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1407/2803] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:33 2022] PUT /identity/v3/projects/bff251349a5f408fbec8f389a96eee2a/users/e37a7019f2f54ce49d9509b016ac99b6/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:33.604876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a80511-8432-4662-80b9-764cf2f90665 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:33.605143 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a80511-8432-4662-80b9-764cf2f90665 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:33.605409 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a80511-8432-4662-80b9-764cf2f90665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:33.619630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:33.637040 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-f5a80511-8432-4662-80b9-764cf2f90665 None None] MFA Rules not processed for user `e37a7019f2f54ce49d9509b016ac99b6`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:33.638186 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f5a80511-8432-4662-80b9-764cf2f90665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:33.708916 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1397/2804] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:33 2022] POST /identity/v3/auth/tokens => generated 3296 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:33.713372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394b7e8-5072-4fb1-a99b-5a128c80e08d admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:33.778399 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8394b7e8-5072-4fb1-a99b-5a128c80e08d admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:33.783217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18ebb5-7ac9-4219-8eb4-bee856e4b253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:33.783952 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18ebb5-7ac9-4219-8eb4-bee856e4b253 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:33.784228 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18ebb5-7ac9-4219-8eb4-bee856e4b253 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:33.784491 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18ebb5-7ac9-4219-8eb4-bee856e4b253 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:33.785126 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18ebb5-7ac9-4219-8eb4-bee856e4b253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:33.787059 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18ebb5-7ac9-4219-8eb4-bee856e4b253 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:33.876440 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1408/2805] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:33 2022] POST /identity/v3/projects => generated 410 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:33.880917 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a80511-8432-4662-80b9-764cf2f90665 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:33.892355 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a80511-8432-4662-80b9-764cf2f90665 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:33.951038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62854d-6db4-4908-8e19-2f811f545846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:33.951849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62854d-6db4-4908-8e19-2f811f545846 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:33.951956 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62854d-6db4-4908-8e19-2f811f545846 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:33.952187 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62854d-6db4-4908-8e19-2f811f545846 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:33.952809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62854d-6db4-4908-8e19-2f811f545846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:33.954734 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a62854d-6db4-4908-8e19-2f811f545846 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:34.068494 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1398/2806] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:33 2022] POST /identity/v3/users => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:34.072528 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18ebb5-7ac9-4219-8eb4-bee856e4b253 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:34.088221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18ebb5-7ac9-4219-8eb4-bee856e4b253 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:34.092935 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52d92e-1cd9-4552-b8d6-f24eef2e6f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:34.093541 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52d92e-1cd9-4552-b8d6-f24eef2e6f28 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:34.093820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52d92e-1cd9-4552-b8d6-f24eef2e6f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:34.094094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52d92e-1cd9-4552-b8d6-f24eef2e6f28 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:34.094754 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52d92e-1cd9-4552-b8d6-f24eef2e6f28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:34.096885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52d92e-1cd9-4552-b8d6-f24eef2e6f28 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:34.108275 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1409/2807] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:34 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:34.112059 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62854d-6db4-4908-8e19-2f811f545846 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:34.123946 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62854d-6db4-4908-8e19-2f811f545846 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:34.128924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31e4ee-3025-4a93-ab5b-4f1b730225dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:34.129606 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e4ee-3025-4a93-ab5b-4f1b730225dc admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:34.129886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e4ee-3025-4a93-ab5b-4f1b730225dc admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:34.130159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31e4ee-3025-4a93-ab5b-4f1b730225dc admin admin] PATH_INFO: `/v3/projects/683a3840640648be87d10fac10f8d5de/users/12cd8f20a9e4454cb8e1d4cca8c1cfb5/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:34.146272 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31e4ee-3025-4a93-ab5b-4f1b730225dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=683a3840640648be87d10fac10f8d5de, user_id=12cd8f20a9e4454cb8e1d4cca8c1cfb5, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:34.148969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31e4ee-3025-4a93-ab5b-4f1b730225dc admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:34.213919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1399/2808] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:34 2022] PUT /identity/v3/projects/683a3840640648be87d10fac10f8d5de/users/12cd8f20a9e4454cb8e1d4cca8c1cfb5/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:12:34.218023 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52d92e-1cd9-4552-b8d6-f24eef2e6f28 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:34.229962 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52d92e-1cd9-4552-b8d6-f24eef2e6f28 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:34.294494 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6861d4f-18cd-4975-8706-aa9aeafb00a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:34.295142 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6861d4f-18cd-4975-8706-aa9aeafb00a4 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:34.295424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6861d4f-18cd-4975-8706-aa9aeafb00a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:34.295721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6861d4f-18cd-4975-8706-aa9aeafb00a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:34.296328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6861d4f-18cd-4975-8706-aa9aeafb00a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:34.297920 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6861d4f-18cd-4975-8706-aa9aeafb00a4 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:34.309759 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1410/2809] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:34 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:34.314026 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31e4ee-3025-4a93-ab5b-4f1b730225dc admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:34.325225 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31e4ee-3025-4a93-ab5b-4f1b730225dc admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:34.329933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a359c95-9346-436d-a7c2-326ae00a9ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:34.331133 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a359c95-9346-436d-a7c2-326ae00a9ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:34.331410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a359c95-9346-436d-a7c2-326ae00a9ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:34.331684 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a359c95-9346-436d-a7c2-326ae00a9ed8 admin admin] PATH_INFO: `/v3/projects/683a3840640648be87d10fac10f8d5de/users/12cd8f20a9e4454cb8e1d4cca8c1cfb5/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:34.333614 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a359c95-9346-436d-a7c2-326ae00a9ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=683a3840640648be87d10fac10f8d5de, user_id=12cd8f20a9e4454cb8e1d4cca8c1cfb5, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:34.336289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a359c95-9346-436d-a7c2-326ae00a9ed8 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:34.347761 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1400/2810] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:34 2022] PUT /identity/v3/projects/683a3840640648be87d10fac10f8d5de/users/12cd8f20a9e4454cb8e1d4cca8c1cfb5/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:34.416577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdbeba-b83b-4f72-a424-c531aa1533e1 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:34.416870 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdbeba-b83b-4f72-a424-c531aa1533e1 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:34.417146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdbeba-b83b-4f72-a424-c531aa1533e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:34.448951 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-dacdbeba-b83b-4f72-a424-c531aa1533e1 None None] MFA Rules not processed for user `12cd8f20a9e4454cb8e1d4cca8c1cfb5`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:34.450238 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-dacdbeba-b83b-4f72-a424-c531aa1533e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:34.485516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:34.532305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1411/2811] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:34 2022] POST /identity/v3/auth/tokens => generated 3363 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:34.543905 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a359c95-9346-436d-a7c2-326ae00a9ed8 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:34.545670 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2a359c95-9346-436d-a7c2-326ae00a9ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:34.602225 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:34.606045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a359c95-9346-436d-a7c2-326ae00a9ed8 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:34.610299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6184ad2-6fe8-4969-96e5-fc9326a2a93a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:34.610915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6184ad2-6fe8-4969-96e5-fc9326a2a93a service cinder] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:34.616099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6184ad2-6fe8-4969-96e5-fc9326a2a93a service cinder] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:34.616099 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6184ad2-6fe8-4969-96e5-fc9326a2a93a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:34.622002 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6184ad2-6fe8-4969-96e5-fc9326a2a93a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:34.623508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6184ad2-6fe8-4969-96e5-fc9326a2a93a service cinder] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:34.634211 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1401/2812] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:34 2022] GET /identity/v3/auth/tokens => generated 3363 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:35.038305 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdbeba-b83b-4f72-a424-c531aa1533e1 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:35.048599 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdbeba-b83b-4f72-a424-c531aa1533e1 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:35.053297 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc5aa7-e33c-4bf5-8137-e51ce692c481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:35.053904 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc5aa7-e33c-4bf5-8137-e51ce692c481 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:35.054439 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc5aa7-e33c-4bf5-8137-e51ce692c481 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:35.054714 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc5aa7-e33c-4bf5-8137-e51ce692c481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:35.116213 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc5aa7-e33c-4bf5-8137-e51ce692c481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:35.117940 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc5aa7-e33c-4bf5-8137-e51ce692c481 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:35.129219 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1412/2813] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:35 2022] GET /identity/v3/auth/tokens => generated 3296 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:43.728662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6184ad2-6fe8-4969-96e5-fc9326a2a93a service cinder] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:43.730182 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-e6184ad2-6fe8-4969-96e5-fc9326a2a93a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:43.822374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6184ad2-6fe8-4969-96e5-fc9326a2a93a service cinder] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:43.827834 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45024d-a454-4c0a-91e1-d11a77c1622d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:43.828525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45024d-a454-4c0a-91e1-d11a77c1622d service placement] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:43.828839 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45024d-a454-4c0a-91e1-d11a77c1622d service placement] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:43.829115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45024d-a454-4c0a-91e1-d11a77c1622d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:43.831618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3e45024d-a454-4c0a-91e1-d11a77c1622d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:43.844307 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45024d-a454-4c0a-91e1-d11a77c1622d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:43.845903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45024d-a454-4c0a-91e1-d11a77c1622d service placement] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:43.857152 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1402/2814] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:43 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:45.123660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=43 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:45.340940 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc5aa7-e33c-4bf5-8137-e51ce692c481 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:45.413741 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc5aa7-e33c-4bf5-8137-e51ce692c481 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:45.418351 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:45.418974 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:45.419284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:45.419555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] PATH_INFO: `/v3/users/e37a7019f2f54ce49d9509b016ac99b6` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:45.420710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e37a7019f2f54ce49d9509b016ac99b6)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:45.422750 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:45.559069 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e37a7019f2f54ce49d9509b016ac99b6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:45.562145 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e37a7019f2f54ce49d9509b016ac99b6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:45.562145 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e37a7019f2f54ce49d9509b016ac99b6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:45.636194 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e37a7019f2f54ce49d9509b016ac99b6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:45.651477 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e37a7019f2f54ce49d9509b016ac99b6'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:45.664644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1413/2815] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:45 2022] DELETE /identity/v3/users/e37a7019f2f54ce49d9509b016ac99b6 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:45.669236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cf8cd-8153-4b23-ab55-2c0e81ab9705 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:45.669765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cf8cd-8153-4b23-ab55-2c0e81ab9705 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:45.669765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cf8cd-8153-4b23-ab55-2c0e81ab9705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:45.689778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-418cf8cd-8153-4b23-ab55-2c0e81ab9705 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:45.691247 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-418cf8cd-8153-4b23-ab55-2c0e81ab9705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:45.754058 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1403/2816] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:45.763990 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:45.765414 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:45.829488 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893d436-38e6-431c-a5b0-a9a811b64cef admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:45.833764 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c378920-b464-410e-be77-cee1d5fad63c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:45.834368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c378920-b464-410e-be77-cee1d5fad63c service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:45.834644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c378920-b464-410e-be77-cee1d5fad63c service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:45.834912 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c378920-b464-410e-be77-cee1d5fad63c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:45.844092 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c378920-b464-410e-be77-cee1d5fad63c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:45.845512 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c378920-b464-410e-be77-cee1d5fad63c service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:45.857258 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1414/2817] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:12:46.311906 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cf8cd-8153-4b23-ab55-2c0e81ab9705 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:46.313412 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-418cf8cd-8153-4b23-ab55-2c0e81ab9705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:46.327259 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418cf8cd-8153-4b23-ab55-2c0e81ab9705 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:46.333285 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:46.333930 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:46.334210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:46.334462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] PATH_INFO: `/v3/projects/bff251349a5f408fbec8f389a96eee2a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:46.335622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff251349a5f408fbec8f389a96eee2a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:46.337525 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:46.341658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c378920-b464-410e-be77-cee1d5fad63c service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:46.343293 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-9c378920-b464-410e-be77-cee1d5fad63c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:46.359094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c378920-b464-410e-be77-cee1d5fad63c service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:46.366727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:46.367473 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:46.367712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:46.367988 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] PATH_INFO: `/v3/users/b08f4e3a7d84483991f0a77dcff02af0` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:46.369154 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08f4e3a7d84483991f0a77dcff02af0)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:46.371180 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:46.390019 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:46.449373 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff251349a5f408fbec8f389a96eee2a', 'request_id': 'req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.456055 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff251349a5f408fbec8f389a96eee2a', 'request_id': 'req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.571249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff251349a5f408fbec8f389a96eee2a', 'request_id': 'req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.584926 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] The token cache is being invalidate because project bff251349a5f408fbec8f389a96eee2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:46.585210 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.585887 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.587494 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1404/2818] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:46 2022] DELETE /identity/v3/projects/bff251349a5f408fbec8f389a96eee2a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:46.591280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:46.592584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:46.599649 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:46.660662 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b9ac3-cc42-4cfc-9d17-0da63ca77793 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:46.665603 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:46.666248 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:46.666521 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:46.666787 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] PATH_INFO: `/v3/users/12cd8f20a9e4454cb8e1d4cca8c1cfb5` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:46.667961 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12cd8f20a9e4454cb8e1d4cca8c1cfb5)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:46.670265 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:46.671464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08f4e3a7d84483991f0a77dcff02af0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.672136 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08f4e3a7d84483991f0a77dcff02af0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.672742 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08f4e3a7d84483991f0a77dcff02af0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.754357 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08f4e3a7d84483991f0a77dcff02af0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.767644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4f2b37ac-3a2d-40ee-a2b9-37fb3f5fd934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08f4e3a7d84483991f0a77dcff02af0'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.782798 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1415/2819] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:46 2022] DELETE /identity/v3/users/b08f4e3a7d84483991f0a77dcff02af0 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:12:46.788182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0479f-1475-411d-85b1-5f7628c8b575 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:46.788526 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0479f-1475-411d-85b1-5f7628c8b575 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:46.788859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b0479f-1475-411d-85b1-5f7628c8b575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:46.809686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-42b0479f-1475-411d-85b1-5f7628c8b575 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:46.812279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-42b0479f-1475-411d-85b1-5f7628c8b575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:46.844714 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12cd8f20a9e4454cb8e1d4cca8c1cfb5'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.845356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12cd8f20a9e4454cb8e1d4cca8c1cfb5'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.846765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12cd8f20a9e4454cb8e1d4cca8c1cfb5'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.875095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1416/2820] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:46.885490 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0479f-1475-411d-85b1-5f7628c8b575 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:46.886922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-42b0479f-1475-411d-85b1-5f7628c8b575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:46.942514 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12cd8f20a9e4454cb8e1d4cca8c1cfb5'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.952000 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b0479f-1475-411d-85b1-5f7628c8b575 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:46.957016 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0301f7-1900-4bcb-9e87-5bbe287733bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:46.960609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0301f7-1900-4bcb-9e87-5bbe287733bd service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:46.960609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0301f7-1900-4bcb-9e87-5bbe287733bd service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:46.960609 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0301f7-1900-4bcb-9e87-5bbe287733bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:46.960996 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12cd8f20a9e4454cb8e1d4cca8c1cfb5'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:46.969040 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0301f7-1900-4bcb-9e87-5bbe287733bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:46.970708 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0301f7-1900-4bcb-9e87-5bbe287733bd service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:46.973376 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1405/2821] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:46 2022] DELETE /identity/v3/users/12cd8f20a9e4454cb8e1d4cca8c1cfb5 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:12:46.985767 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1417/2822] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:12:47.514569 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:47.515932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:47.520745 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0301f7-1900-4bcb-9e87-5bbe287733bd service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:47.523048 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-bb0301f7-1900-4bcb-9e87-5bbe287733bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:47.541813 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0301f7-1900-4bcb-9e87-5bbe287733bd service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:47.594616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b264ae-03b6-4449-9172-18097f436c80 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:47.600047 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:47.600696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:47.600974 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:47.601249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] PATH_INFO: `/v3/projects/683a3840640648be87d10fac10f8d5de` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:47.601653 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:47.602391 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=683a3840640648be87d10fac10f8d5de)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:47.604419 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:47.626791 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:47.628148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:47.628426 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:47.628694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] PATH_INFO: `/v3/projects/d4ef92db2d3b4157913b1d428ecdbb1e` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:47.630070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4ef92db2d3b4157913b1d428ecdbb1e)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:47.632038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:47.734969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:47.753643 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683a3840640648be87d10fac10f8d5de', 'request_id': 'req-10177748-54f5-4a56-ba4d-25f066e49183'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:47.759865 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683a3840640648be87d10fac10f8d5de', 'request_id': 'req-10177748-54f5-4a56-ba4d-25f066e49183'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:47.801721 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4ef92db2d3b4157913b1d428ecdbb1e', 'request_id': 'req-4c782019-9ea8-486f-b2b6-77b5ab10e108'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:47.808726 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4ef92db2d3b4157913b1d428ecdbb1e', 'request_id': 'req-4c782019-9ea8-486f-b2b6-77b5ab10e108'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:47.814605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4ef92db2d3b4157913b1d428ecdbb1e', 'request_id': 'req-4c782019-9ea8-486f-b2b6-77b5ab10e108'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:47.841220 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683a3840640648be87d10fac10f8d5de', 'request_id': 'req-10177748-54f5-4a56-ba4d-25f066e49183'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:47.852590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] The token cache is being invalidate because project 683a3840640648be87d10fac10f8d5de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:47.852876 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:47.853518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-10177748-54f5-4a56-ba4d-25f066e49183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:47.855000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1406/2823] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:47 2022] DELETE /identity/v3/projects/683a3840640648be87d10fac10f8d5de => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:47.866774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50e069-e3d5-4840-9ecd-7398a25c3e71 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:47.867161 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50e069-e3d5-4840-9ecd-7398a25c3e71 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:47.867438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50e069-e3d5-4840-9ecd-7398a25c3e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:47.888922 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-0e50e069-e3d5-4840-9ecd-7398a25c3e71 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:47.896241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-0e50e069-e3d5-4840-9ecd-7398a25c3e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:47.926622 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] The token cache is being invalidate because project d4ef92db2d3b4157913b1d428ecdbb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:47.926910 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:47.930924 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-4c782019-9ea8-486f-b2b6-77b5ab10e108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:47.933104 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1418/2824] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:47 2022] DELETE /identity/v3/projects/d4ef92db2d3b4157913b1d428ecdbb1e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:47.938288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d7f8b-f330-42a3-8e9c-a2f66d14d50f None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:47.938572 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d7f8b-f330-42a3-8e9c-a2f66d14d50f None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:47.938859 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d7f8b-f330-42a3-8e9c-a2f66d14d50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:47.955278 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1407/2825] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:47.960809 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-f56d7f8b-f330-42a3-8e9c-a2f66d14d50f None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:47.962038 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f56d7f8b-f330-42a3-8e9c-a2f66d14d50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:47.964295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e3052-665a-4e35-ab16-82c26286db37 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:47.965064 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e3052-665a-4e35-ab16-82c26286db37 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:47.965225 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e3052-665a-4e35-ab16-82c26286db37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:47.970825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1419/2826] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:47.980329 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018daf7c-2481-4b8c-829d-dd9e5315300e None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:47.983256 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018daf7c-2481-4b8c-829d-dd9e5315300e None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:47.983256 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018daf7c-2481-4b8c-829d-dd9e5315300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:47.988340 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-5b6e3052-665a-4e35-ab16-82c26286db37 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:47.989747 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-5b6e3052-665a-4e35-ab16-82c26286db37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:48.002402 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1408/2827] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:48.004391 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-018daf7c-2481-4b8c-829d-dd9e5315300e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:48.004651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e3052-665a-4e35-ab16-82c26286db37 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.005603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-018daf7c-2481-4b8c-829d-dd9e5315300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:48.014179 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1420/2828] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:48.016942 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e3052-665a-4e35-ab16-82c26286db37 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.018913 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018daf7c-2481-4b8c-829d-dd9e5315300e None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.022025 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bffff-3fb6-445b-b48d-80bc9ddc7cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.022631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bffff-3fb6-445b-b48d-80bc9ddc7cdc admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.022973 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bffff-3fb6-445b-b48d-80bc9ddc7cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.023308 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bffff-3fb6-445b-b48d-80bc9ddc7cdc admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.023962 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bffff-3fb6-445b-b48d-80bc9ddc7cdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:12:48.024261 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bffff-3fb6-445b-b48d-80bc9ddc7cdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.025858 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bffff-3fb6-445b-b48d-80bc9ddc7cdc admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.028694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018daf7c-2481-4b8c-829d-dd9e5315300e None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.033516 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c6c4c-362f-4968-a3d0-1b40ea73379f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.034201 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c6c4c-362f-4968-a3d0-1b40ea73379f admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.034480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c6c4c-362f-4968-a3d0-1b40ea73379f admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.034820 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c6c4c-362f-4968-a3d0-1b40ea73379f admin admin] PATH_INFO: `/v3/domains` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.037644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c6c4c-362f-4968-a3d0-1b40ea73379f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82575) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:12:48.037644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c6c4c-362f-4968-a3d0-1b40ea73379f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.037644 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c6c4c-362f-4968-a3d0-1b40ea73379f admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.043090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1409/2829] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:12:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:12:48.049972 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bffff-3fb6-445b-b48d-80bc9ddc7cdc admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.052373 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1421/2830] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:12:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:12:48.056927 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c6c4c-362f-4968-a3d0-1b40ea73379f admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.061678 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bffff-3fb6-445b-b48d-80bc9ddc7cdc admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.069252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d466aa-7869-45f3-97c0-ae54a478c073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.069965 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d466aa-7869-45f3-97c0-ae54a478c073 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.070245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d466aa-7869-45f3-97c0-ae54a478c073 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.070518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d466aa-7869-45f3-97c0-ae54a478c073 admin admin] PATH_INFO: `/v3/projects` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.070658 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c6c4c-362f-4968-a3d0-1b40ea73379f admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.071179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d466aa-7869-45f3-97c0-ae54a478c073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.073110 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d466aa-7869-45f3-97c0-ae54a478c073 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.076340 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ded3f-0e8d-44e6-a1ed-92c910a853f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.080916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ded3f-0e8d-44e6-a1ed-92c910a853f2 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.080916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ded3f-0e8d-44e6-a1ed-92c910a853f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.080916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488ded3f-0e8d-44e6-a1ed-92c910a853f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.080916 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ded3f-0e8d-44e6-a1ed-92c910a853f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.082074 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488ded3f-0e8d-44e6-a1ed-92c910a853f2 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.168924 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1410/2831] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:48 2022] POST /identity/v3/projects => generated 370 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:48.173230 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d466aa-7869-45f3-97c0-ae54a478c073 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.226932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1422/2832] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:48 2022] POST /identity/v3/projects => generated 384 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:48.231261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ded3f-0e8d-44e6-a1ed-92c910a853f2 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.250432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d466aa-7869-45f3-97c0-ae54a478c073 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.269737 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:48.308710 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488ded3f-0e8d-44e6-a1ed-92c910a853f2 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.313666 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983d72-cd02-4010-9b35-e7698896426a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.314279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983d72-cd02-4010-9b35-e7698896426a admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.314551 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983d72-cd02-4010-9b35-e7698896426a admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.314814 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13983d72-cd02-4010-9b35-e7698896426a admin admin] PATH_INFO: `/v3/users` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.316770 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13983d72-cd02-4010-9b35-e7698896426a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.317989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b11f8-5840-4671-ad7a-ad2044bee5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.318654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b11f8-5840-4671-ad7a-ad2044bee5ba admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.318768 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13983d72-cd02-4010-9b35-e7698896426a admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.318951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b11f8-5840-4671-ad7a-ad2044bee5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.319249 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b11f8-5840-4671-ad7a-ad2044bee5ba admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.319877 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b11f8-5840-4671-ad7a-ad2044bee5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.321871 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b11f8-5840-4671-ad7a-ad2044bee5ba admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.426112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1423/2833] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:48 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:48.431044 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983d72-cd02-4010-9b35-e7698896426a admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.442549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13983d72-cd02-4010-9b35-e7698896426a admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.443628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1411/2834] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:48 2022] POST /identity/v3/users => generated 286 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:12:48.447730 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b11f8-5840-4671-ad7a-ad2044bee5ba admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.450430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee934565-a344-4c9e-8bc6-12b89089a5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.451076 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee934565-a344-4c9e-8bc6-12b89089a5c3 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.451359 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee934565-a344-4c9e-8bc6-12b89089a5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.451676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee934565-a344-4c9e-8bc6-12b89089a5c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.452458 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee934565-a344-4c9e-8bc6-12b89089a5c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.454060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee934565-a344-4c9e-8bc6-12b89089a5c3 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.457562 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:48.459931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b11f8-5840-4671-ad7a-ad2044bee5ba admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.465089 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d11146-76bb-4352-81b0-18235b73e472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.465681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d11146-76bb-4352-81b0-18235b73e472 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.465957 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d11146-76bb-4352-81b0-18235b73e472 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.466226 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d11146-76bb-4352-81b0-18235b73e472 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.466396 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1424/2835] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:48 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:12:48.466872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d11146-76bb-4352-81b0-18235b73e472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.468573 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d11146-76bb-4352-81b0-18235b73e472 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.475942 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee934565-a344-4c9e-8bc6-12b89089a5c3 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.485607 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1412/2836] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:48 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:12:48.487374 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d11146-76bb-4352-81b0-18235b73e472 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.489352 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee934565-a344-4c9e-8bc6-12b89089a5c3 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.496341 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc6b7e-d58e-4b3e-ad42-6a2de5f4c0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.497298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc6b7e-d58e-4b3e-ad42-6a2de5f4c0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.497765 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc6b7e-d58e-4b3e-ad42-6a2de5f4c0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.498032 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d11146-76bb-4352-81b0-18235b73e472 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.498157 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc6b7e-d58e-4b3e-ad42-6a2de5f4c0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.499112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc6b7e-d58e-4b3e-ad42-6a2de5f4c0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.501909 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc6b7e-d58e-4b3e-ad42-6a2de5f4c0d7 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.503523 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223777f8-28f7-453f-81fa-1773cef2ecbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.509157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223777f8-28f7-453f-81fa-1773cef2ecbc admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.509157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223777f8-28f7-453f-81fa-1773cef2ecbc admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.509157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223777f8-28f7-453f-81fa-1773cef2ecbc admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.509157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223777f8-28f7-453f-81fa-1773cef2ecbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.509157 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223777f8-28f7-453f-81fa-1773cef2ecbc admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.519686 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1413/2837] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:48 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:48.523214 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1425/2838] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:48 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:12:48.524177 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223777f8-28f7-453f-81fa-1773cef2ecbc admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.526295 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc6b7e-d58e-4b3e-ad42-6a2de5f4c0d7 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.535224 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223777f8-28f7-453f-81fa-1773cef2ecbc admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.538479 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc6b7e-d58e-4b3e-ad42-6a2de5f4c0d7 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.540620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c91a5-6578-4522-934f-914340d5d075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.541275 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c91a5-6578-4522-934f-914340d5d075 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.541549 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c91a5-6578-4522-934f-914340d5d075 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.541817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c91a5-6578-4522-934f-914340d5d075 admin admin] PATH_INFO: `/v3/projects/382c3868ca9d415c9b7157f4fdc29c30/users/dd01c9ee077b49c6820ecc2c6ee94046/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.544169 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686d329-ef0d-44bf-9e68-b530be2cc110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.544817 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686d329-ef0d-44bf-9e68-b530be2cc110 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.545099 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686d329-ef0d-44bf-9e68-b530be2cc110 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.545372 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686d329-ef0d-44bf-9e68-b530be2cc110 admin admin] PATH_INFO: `/v3/projects/97de78ef033344cab1c3d834903cba0e/users/88cf69eb3ae84fe88a5bc6cb162a75cd/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.560903 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c91a5-6578-4522-934f-914340d5d075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=382c3868ca9d415c9b7157f4fdc29c30, user_id=dd01c9ee077b49c6820ecc2c6ee94046, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.562148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686d329-ef0d-44bf-9e68-b530be2cc110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97de78ef033344cab1c3d834903cba0e, user_id=88cf69eb3ae84fe88a5bc6cb162a75cd, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.563833 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c91a5-6578-4522-934f-914340d5d075 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.564850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686d329-ef0d-44bf-9e68-b530be2cc110 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.665860 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1414/2839] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:48 2022] PUT /identity/v3/projects/382c3868ca9d415c9b7157f4fdc29c30/users/dd01c9ee077b49c6820ecc2c6ee94046/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:12:48.679538 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2255ed5-2b9d-4981-9486-41aa6714163b None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.680189 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2255ed5-2b9d-4981-9486-41aa6714163b None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.680462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2255ed5-2b9d-4981-9486-41aa6714163b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.707108 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1426/2840] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:48 2022] PUT /identity/v3/projects/97de78ef033344cab1c3d834903cba0e/users/88cf69eb3ae84fe88a5bc6cb162a75cd/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:12:48.711553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b2255ed5-2b9d-4981-9486-41aa6714163b None None] MFA Rules not processed for user `dd01c9ee077b49c6820ecc2c6ee94046`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:48.712937 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b2255ed5-2b9d-4981-9486-41aa6714163b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:48.719316 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499ea8d-8b2b-471f-b017-ee5716d760bb None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.719895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499ea8d-8b2b-471f-b017-ee5716d760bb None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.720557 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1499ea8d-8b2b-471f-b017-ee5716d760bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.756327 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-1499ea8d-8b2b-471f-b017-ee5716d760bb None None] MFA Rules not processed for user `88cf69eb3ae84fe88a5bc6cb162a75cd`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:48.757561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1499ea8d-8b2b-471f-b017-ee5716d760bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:48.789045 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1415/2841] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:48.800583 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2255ed5-2b9d-4981-9486-41aa6714163b None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.802053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b2255ed5-2b9d-4981-9486-41aa6714163b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:48.842222 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1427/2842] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:12:48.856547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499ea8d-8b2b-471f-b017-ee5716d760bb None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:48.858734 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1499ea8d-8b2b-471f-b017-ee5716d760bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:48.886751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2255ed5-2b9d-4981-9486-41aa6714163b None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.891283 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587d055-5910-42a7-b43a-4dfd3498b6d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f1d62a4c07c4402ea5803f6e68fb1d0b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.892035 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d055-5910-42a7-b43a-4dfd3498b6d9 service glance] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.892474 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d055-5910-42a7-b43a-4dfd3498b6d9 service glance] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.892943 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d055-5910-42a7-b43a-4dfd3498b6d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.902797 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:48.906707 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587d055-5910-42a7-b43a-4dfd3498b6d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.908783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587d055-5910-42a7-b43a-4dfd3498b6d9 service glance] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.918920 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1416/2843] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:12:48.939328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1499ea8d-8b2b-471f-b017-ee5716d760bb None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:48.947239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25825d4-7ea6-4599-8eeb-2dcbac9ec27d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:48.947239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25825d4-7ea6-4599-8eeb-2dcbac9ec27d service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:48.947239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25825d4-7ea6-4599-8eeb-2dcbac9ec27d service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:48.947239 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25825d4-7ea6-4599-8eeb-2dcbac9ec27d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:48.954984 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25825d4-7ea6-4599-8eeb-2dcbac9ec27d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:48.957088 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25825d4-7ea6-4599-8eeb-2dcbac9ec27d service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:48.969407 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1428/2844] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:12:53.189654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587d055-5910-42a7-b43a-4dfd3498b6d9 service glance] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:53.263438 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587d055-5910-42a7-b43a-4dfd3498b6d9 service glance] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:53.267907 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:53.269086 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:53.269360 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:53.269632 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] PATH_INFO: `/v3/users/88cf69eb3ae84fe88a5bc6cb162a75cd` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:53.270772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=88cf69eb3ae84fe88a5bc6cb162a75cd)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:53.272742 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:53.454725 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88cf69eb3ae84fe88a5bc6cb162a75cd'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:53.455440 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88cf69eb3ae84fe88a5bc6cb162a75cd'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:53.456315 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88cf69eb3ae84fe88a5bc6cb162a75cd'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:53.510752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88cf69eb3ae84fe88a5bc6cb162a75cd'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:53.530092 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88cf69eb3ae84fe88a5bc6cb162a75cd'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:53.543497 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1417/2845] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:12:53 2022] DELETE /identity/v3/users/88cf69eb3ae84fe88a5bc6cb162a75cd => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:53.548603 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e0b7e-9117-420d-bc86-34c1f45453b8 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:53.549070 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e0b7e-9117-420d-bc86-34c1f45453b8 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:53.549193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e0b7e-9117-420d-bc86-34c1f45453b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:53.569456 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-cc9e0b7e-9117-420d-bc86-34c1f45453b8 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:53.570724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cc9e0b7e-9117-420d-bc86-34c1f45453b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:53.636355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:53.648634 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1429/2846] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:53.658429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:53.660770 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:53.727740 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f3c437-bfcf-49e7-931e-5a60bbaf98ef admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:53.732252 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392654f9-4923-45e2-8607-ec34ec6c9a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:53.732899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392654f9-4923-45e2-8607-ec34ec6c9a88 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:53.733179 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392654f9-4923-45e2-8607-ec34ec6c9a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:53.733450 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392654f9-4923-45e2-8607-ec34ec6c9a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:53.744022 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392654f9-4923-45e2-8607-ec34ec6c9a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:53.745535 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392654f9-4923-45e2-8607-ec34ec6c9a88 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:53.757170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1418/2847] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:12:54.329782 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e0b7e-9117-420d-bc86-34c1f45453b8 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:54.335416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-cc9e0b7e-9117-420d-bc86-34c1f45453b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:54.352225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e0b7e-9117-420d-bc86-34c1f45453b8 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:54.360434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:54.361470 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:54.361846 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:54.362966 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] PATH_INFO: `/v3/projects/97de78ef033344cab1c3d834903cba0e` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:54.364417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97de78ef033344cab1c3d834903cba0e)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:54.366422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:54.480828 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97de78ef033344cab1c3d834903cba0e', 'request_id': 'req-6da65696-75ba-4f8f-b8fb-1455970b3d82'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:54.488896 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97de78ef033344cab1c3d834903cba0e', 'request_id': 'req-6da65696-75ba-4f8f-b8fb-1455970b3d82'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:54.495713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97de78ef033344cab1c3d834903cba0e', 'request_id': 'req-6da65696-75ba-4f8f-b8fb-1455970b3d82'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:54.567785 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] The token cache is being invalidate because project 97de78ef033344cab1c3d834903cba0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:12:54.568922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:54.569217 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-6da65696-75ba-4f8f-b8fb-1455970b3d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:12:54.571970 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1430/2848] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:12:54 2022] DELETE /identity/v3/projects/97de78ef033344cab1c3d834903cba0e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:54.582306 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2dc9a-229c-495b-9fd9-1055a5f5255b None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:54.583063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2dc9a-229c-495b-9fd9-1055a5f5255b None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:54.583063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2dc9a-229c-495b-9fd9-1055a5f5255b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:54.617638 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-92c2dc9a-229c-495b-9fd9-1055a5f5255b None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:54.618898 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-92c2dc9a-229c-495b-9fd9-1055a5f5255b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:54.632618 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:12:54.713184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1419/2849] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:54.722298 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ae60c-2c6f-4c8c-8b76-82f85ed06beb None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:54.722573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ae60c-2c6f-4c8c-8b76-82f85ed06beb None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:54.726580 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ae60c-2c6f-4c8c-8b76-82f85ed06beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:54.744361 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-4d9ae60c-2c6f-4c8c-8b76-82f85ed06beb None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:54.745596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4d9ae60c-2c6f-4c8c-8b76-82f85ed06beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:54.756461 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1431/2850] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:54.761027 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2dc9a-229c-495b-9fd9-1055a5f5255b None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:54.771872 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2dc9a-229c-495b-9fd9-1055a5f5255b None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:54.777038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f5986-76f4-4f63-bbf4-ebb47a1fc95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:54.777657 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f5986-76f4-4f63-bbf4-ebb47a1fc95e admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:54.777933 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f5986-76f4-4f63-bbf4-ebb47a1fc95e admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:54.778203 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f5986-76f4-4f63-bbf4-ebb47a1fc95e admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:54.778869 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f5986-76f4-4f63-bbf4-ebb47a1fc95e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:12:54.779204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f5986-76f4-4f63-bbf4-ebb47a1fc95e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:54.780822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f5986-76f4-4f63-bbf4-ebb47a1fc95e admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:54.796030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1420/2851] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:12:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:12:54.800699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ae60c-2c6f-4c8c-8b76-82f85ed06beb None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:54.810762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ae60c-2c6f-4c8c-8b76-82f85ed06beb None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:54.815824 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae9af-1b99-408f-8496-fe00f4eaf5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:54.816431 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae9af-1b99-408f-8496-fe00f4eaf5ff admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:54.816712 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae9af-1b99-408f-8496-fe00f4eaf5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:54.816983 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae9af-1b99-408f-8496-fe00f4eaf5ff admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:54.817751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae9af-1b99-408f-8496-fe00f4eaf5ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:54.819591 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae9af-1b99-408f-8496-fe00f4eaf5ff admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:54.925616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1432/2852] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:12:54 2022] POST /identity/v3/projects => generated 392 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:12:54.929701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f5986-76f4-4f63-bbf4-ebb47a1fc95e admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:54.946432 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f5986-76f4-4f63-bbf4-ebb47a1fc95e admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:55.012405 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74694a7-3a54-4a70-a514-23753e709855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:55.013074 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74694a7-3a54-4a70-a514-23753e709855 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:55.013358 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74694a7-3a54-4a70-a514-23753e709855 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:55.013629 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74694a7-3a54-4a70-a514-23753e709855 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:55.014290 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74694a7-3a54-4a70-a514-23753e709855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:55.016309 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74694a7-3a54-4a70-a514-23753e709855 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:55.098480 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1421/2853] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:12:54 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:12:55.102728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae9af-1b99-408f-8496-fe00f4eaf5ff admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:55.114921 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae9af-1b99-408f-8496-fe00f4eaf5ff admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:55.119995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce6977-35da-4450-8882-67130b07e7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:55.120660 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce6977-35da-4450-8882-67130b07e7bb admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:55.120937 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce6977-35da-4450-8882-67130b07e7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:55.121205 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce6977-35da-4450-8882-67130b07e7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:55.121886 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce6977-35da-4450-8882-67130b07e7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:55.123527 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ce6977-35da-4450-8882-67130b07e7bb admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:55.138607 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1433/2854] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:55 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:12:55.141371 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74694a7-3a54-4a70-a514-23753e709855 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:55.156428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74694a7-3a54-4a70-a514-23753e709855 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:55.163695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669534b9-21bb-458b-a067-896bcf80e6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:55.164637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669534b9-21bb-458b-a067-896bcf80e6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:55.165083 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669534b9-21bb-458b-a067-896bcf80e6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:55.165520 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669534b9-21bb-458b-a067-896bcf80e6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:55.166547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669534b9-21bb-458b-a067-896bcf80e6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:55.169063 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669534b9-21bb-458b-a067-896bcf80e6d5 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:55.189923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1422/2855] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:12:55 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:12:55.191684 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce6977-35da-4450-8882-67130b07e7bb admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:55.204206 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce6977-35da-4450-8882-67130b07e7bb admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:55.209617 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77557130-7855-4bf1-9e81-e5381ca415b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:55.210276 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77557130-7855-4bf1-9e81-e5381ca415b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:55.210555 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77557130-7855-4bf1-9e81-e5381ca415b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:55.210825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77557130-7855-4bf1-9e81-e5381ca415b2 admin admin] PATH_INFO: `/v3/projects/c92daea5e3744befa6d73aeb380891f4/users/64d18ab1a1814a518ba12446169abd43/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:55.226401 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77557130-7855-4bf1-9e81-e5381ca415b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c92daea5e3744befa6d73aeb380891f4, user_id=64d18ab1a1814a518ba12446169abd43, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:55.230141 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77557130-7855-4bf1-9e81-e5381ca415b2 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:55.323649 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1434/2856] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:12:55 2022] PUT /identity/v3/projects/c92daea5e3744befa6d73aeb380891f4/users/64d18ab1a1814a518ba12446169abd43/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:12:55.335375 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb6c08-d9c9-4f1d-9e1b-7275cd0b1683 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:55.335751 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb6c08-d9c9-4f1d-9e1b-7275cd0b1683 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:55.335890 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb6c08-d9c9-4f1d-9e1b-7275cd0b1683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:55.366262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9bdb6c08-d9c9-4f1d-9e1b-7275cd0b1683 None None] MFA Rules not processed for user `64d18ab1a1814a518ba12446169abd43`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:12:55.367548 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9bdb6c08-d9c9-4f1d-9e1b-7275cd0b1683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:55.444269 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1423/2857] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:12:55 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:12:55.455646 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77557130-7855-4bf1-9e81-e5381ca415b2 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:12:55.458204 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-77557130-7855-4bf1-9e81-e5381ca415b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:12:55.531700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77557130-7855-4bf1-9e81-e5381ca415b2 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:12:55.535892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd057094-a0af-4247-be38-a3ff04770312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:12:55.536525 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd057094-a0af-4247-be38-a3ff04770312 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:12:55.536923 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd057094-a0af-4247-be38-a3ff04770312 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:12:55.537202 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd057094-a0af-4247-be38-a3ff04770312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:12:55.546482 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd057094-a0af-4247-be38-a3ff04770312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:12:55.547982 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd057094-a0af-4247-be38-a3ff04770312 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:12:55.561457 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1435/2858] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:12:55 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:13:05.439808 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:13:05.556435 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:13:26.960817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb6c08-d9c9-4f1d-9e1b-7275cd0b1683 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:27.051225 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb6c08-d9c9-4f1d-9e1b-7275cd0b1683 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:27.057929 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:27.062634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:27.062634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:27.062634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] PATH_INFO: `/v3/users/64d18ab1a1814a518ba12446169abd43` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:27.062634 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64d18ab1a1814a518ba12446169abd43)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:27.064291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:27.312043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64d18ab1a1814a518ba12446169abd43'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:27.312710 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64d18ab1a1814a518ba12446169abd43'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:27.313408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64d18ab1a1814a518ba12446169abd43'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:27.390654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64d18ab1a1814a518ba12446169abd43'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:27.409932 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64d18ab1a1814a518ba12446169abd43'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:27.426455 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1424/2859] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:13:26 2022] DELETE /identity/v3/users/64d18ab1a1814a518ba12446169abd43 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:13:27.432368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296878-c51b-4895-b636-b62fb7ab5959 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:27.432853 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296878-c51b-4895-b636-b62fb7ab5959 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:27.432972 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe296878-c51b-4895-b636-b62fb7ab5959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:27.461020 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-fe296878-c51b-4895-b636-b62fb7ab5959 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:13:27.462654 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fe296878-c51b-4895-b636-b62fb7ab5959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:27.549775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1436/2860] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:13:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:13:27.561624 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:27.565530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:27.628931 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609f87b-ed10-45a5-ab6b-68c6ec60db63 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:27.632993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed88788-a99e-4084-8ddc-4bbdda62323f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:27.633623 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88788-a99e-4084-8ddc-4bbdda62323f service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:27.633935 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88788-a99e-4084-8ddc-4bbdda62323f service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:27.634410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed88788-a99e-4084-8ddc-4bbdda62323f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:27.645043 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed88788-a99e-4084-8ddc-4bbdda62323f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:27.646731 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed88788-a99e-4084-8ddc-4bbdda62323f service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:27.658141 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1425/2861] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:13:27 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:13:28.137232 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe296878-c51b-4895-b636-b62fb7ab5959 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:28.138670 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fe296878-c51b-4895-b636-b62fb7ab5959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:28.152186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe296878-c51b-4895-b636-b62fb7ab5959 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:28.158469 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:28.159150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:28.159434 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:28.159843 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] PATH_INFO: `/v3/projects/c92daea5e3744befa6d73aeb380891f4` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:28.164031 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c92daea5e3744befa6d73aeb380891f4)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:28.166260 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:28.283900 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c92daea5e3744befa6d73aeb380891f4', 'request_id': 'req-fce0160a-29c9-4350-b980-bc1ee2cff764'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:28.290486 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c92daea5e3744befa6d73aeb380891f4', 'request_id': 'req-fce0160a-29c9-4350-b980-bc1ee2cff764'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:28.296096 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c92daea5e3744befa6d73aeb380891f4', 'request_id': 'req-fce0160a-29c9-4350-b980-bc1ee2cff764'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:28.351567 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] The token cache is being invalidate because project c92daea5e3744befa6d73aeb380891f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:13:28.351841 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:28.352476 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-fce0160a-29c9-4350-b980-bc1ee2cff764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:28.354015 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1437/2862] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:13:28 2022] DELETE /identity/v3/projects/c92daea5e3744befa6d73aeb380891f4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:13:28.358967 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636fe33-5291-4498-b4dd-126f144c174e None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:28.359510 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636fe33-5291-4498-b4dd-126f144c174e None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:28.359637 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636fe33-5291-4498-b4dd-126f144c174e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:28.382169 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-1636fe33-5291-4498-b4dd-126f144c174e None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:13:28.383325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-1636fe33-5291-4498-b4dd-126f144c174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:28.401223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:13:28.447184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1426/2863] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:13:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:13:28.455063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1893c-8de4-4995-b59a-bcd4dae978a1 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:28.455521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1893c-8de4-4995-b59a-bcd4dae978a1 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:28.455521 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1893c-8de4-4995-b59a-bcd4dae978a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:28.479597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-4aa1893c-8de4-4995-b59a-bcd4dae978a1 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:13:28.479597 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4aa1893c-8de4-4995-b59a-bcd4dae978a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:28.486618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1438/2864] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:13:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:13:28.491752 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636fe33-5291-4498-b4dd-126f144c174e None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:28.504630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636fe33-5291-4498-b4dd-126f144c174e None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:28.509410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117eeb0f-0729-4d7d-8b65-143009341bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:28.510020 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117eeb0f-0729-4d7d-8b65-143009341bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:28.510295 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117eeb0f-0729-4d7d-8b65-143009341bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:28.510622 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117eeb0f-0729-4d7d-8b65-143009341bb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:28.511238 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117eeb0f-0729-4d7d-8b65-143009341bb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=82574) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:13:28.511534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117eeb0f-0729-4d7d-8b65-143009341bb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:28.513127 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117eeb0f-0729-4d7d-8b65-143009341bb3 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:28.525493 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1427/2865] 10.222.0.105 () {58 vars in 1158 bytes} [Wed Aug 24 05:13:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:13:28.530223 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1893c-8de4-4995-b59a-bcd4dae978a1 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:28.540700 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1893c-8de4-4995-b59a-bcd4dae978a1 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:28.545683 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b22cd4-3fce-442f-b39c-dc8315476c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:28.548208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b22cd4-3fce-442f-b39c-dc8315476c28 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:28.548208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b22cd4-3fce-442f-b39c-dc8315476c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:28.548208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b22cd4-3fce-442f-b39c-dc8315476c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:28.548208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b22cd4-3fce-442f-b39c-dc8315476c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:28.549484 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b22cd4-3fce-442f-b39c-dc8315476c28 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:28.637790 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1439/2866] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:13:28 2022] POST /identity/v3/projects => generated 380 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:13:28.642501 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117eeb0f-0729-4d7d-8b65-143009341bb3 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:28.652960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117eeb0f-0729-4d7d-8b65-143009341bb3 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:28.719240 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a401837-715e-4410-bdde-4b6f69578f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:28.719897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a401837-715e-4410-bdde-4b6f69578f04 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:28.720155 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a401837-715e-4410-bdde-4b6f69578f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:28.720422 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a401837-715e-4410-bdde-4b6f69578f04 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:28.721053 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a401837-715e-4410-bdde-4b6f69578f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:28.723936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a401837-715e-4410-bdde-4b6f69578f04 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:28.831046 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1428/2867] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:13:28 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:13:28.836805 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b22cd4-3fce-442f-b39c-dc8315476c28 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:28.849353 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b22cd4-3fce-442f-b39c-dc8315476c28 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:28.854095 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da80a21-a616-4ade-83cf-b4142f126819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:28.854704 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da80a21-a616-4ade-83cf-b4142f126819 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:28.854978 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da80a21-a616-4ade-83cf-b4142f126819 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:28.855271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da80a21-a616-4ade-83cf-b4142f126819 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:28.855907 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da80a21-a616-4ade-83cf-b4142f126819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:28.857520 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da80a21-a616-4ade-83cf-b4142f126819 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:28.868727 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1440/2868] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:13:28 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:13:28.873132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a401837-715e-4410-bdde-4b6f69578f04 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:28.884630 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a401837-715e-4410-bdde-4b6f69578f04 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:28.889697 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2a6a61-304f-4dca-b373-36ecd0d5e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:28.890280 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a6a61-304f-4dca-b373-36ecd0d5e3af admin admin] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:28.890556 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a6a61-304f-4dca-b373-36ecd0d5e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:28.890824 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2a6a61-304f-4dca-b373-36ecd0d5e3af admin admin] PATH_INFO: `/v3/roles` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:28.891567 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a6a61-304f-4dca-b373-36ecd0d5e3af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:28.893162 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2a6a61-304f-4dca-b373-36ecd0d5e3af admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:28.907456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1429/2869] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:13:28 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:13:28.913618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da80a21-a616-4ade-83cf-b4142f126819 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:28.924125 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da80a21-a616-4ade-83cf-b4142f126819 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:28.929784 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20246099-e06e-486a-9138-27204be465bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:28.930478 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20246099-e06e-486a-9138-27204be465bf admin admin] REQUEST_METHOD: `PUT` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:28.930758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20246099-e06e-486a-9138-27204be465bf admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:28.931057 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20246099-e06e-486a-9138-27204be465bf admin admin] PATH_INFO: `/v3/projects/2a010cb56074448a979b3b7d944a92d6/users/11c7bb2f319e4bd2a7e03a4f77d29c4f/roles/df27fa65390e4de78fa117773dc73555` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:28.949009 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20246099-e06e-486a-9138-27204be465bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a010cb56074448a979b3b7d944a92d6, user_id=11c7bb2f319e4bd2a7e03a4f77d29c4f, role_id=df27fa65390e4de78fa117773dc73555)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:28.951716 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20246099-e06e-486a-9138-27204be465bf admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:29.049304 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1441/2870] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:13:28 2022] PUT /identity/v3/projects/2a010cb56074448a979b3b7d944a92d6/users/11c7bb2f319e4bd2a7e03a4f77d29c4f/roles/df27fa65390e4de78fa117773dc73555 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:13:29.054715 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f8dc1-12aa-4e26-b87e-425e648025c2 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:29.054999 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f8dc1-12aa-4e26-b87e-425e648025c2 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:29.055299 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f8dc1-12aa-4e26-b87e-425e648025c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:29.075294 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-9e7f8dc1-12aa-4e26-b87e-425e648025c2 None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:13:29.076546 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-9e7f8dc1-12aa-4e26-b87e-425e648025c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:29.142370 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1430/2871] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:13:29 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:13:29.153041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20246099-e06e-486a-9138-27204be465bf admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:29.154825 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-20246099-e06e-486a-9138-27204be465bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:29.179697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:13:29.217550 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20246099-e06e-486a-9138-27204be465bf admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:29.221348 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8294f0-c45c-4ce8-91b8-7030a5b612b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:29.225546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8294f0-c45c-4ce8-91b8-7030a5b612b4 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:29.225546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8294f0-c45c-4ce8-91b8-7030a5b612b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:29.225546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8294f0-c45c-4ce8-91b8-7030a5b612b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:29.232657 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8294f0-c45c-4ce8-91b8-7030a5b612b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:29.234094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8294f0-c45c-4ce8-91b8-7030a5b612b4 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:29.245885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1442/2872] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:13:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:13:35.016596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999764be-1727-4c30-8fb6-d730ec92517f None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:35.017235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999764be-1727-4c30-8fb6-d730ec92517f None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:35.017235 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999764be-1727-4c30-8fb6-d730ec92517f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:35.058581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-999764be-1727-4c30-8fb6-d730ec92517f None None] MFA Rules not processed for user `11c7bb2f319e4bd2a7e03a4f77d29c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:13:35.060822 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-999764be-1727-4c30-8fb6-d730ec92517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:35.092410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:13:35.156918 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1431/2873] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:13:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:13:35.162128 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8294f0-c45c-4ce8-91b8-7030a5b612b4 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:35.173099 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8294f0-c45c-4ce8-91b8-7030a5b612b4 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:35.178981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba35b3-b5e1-421a-84b8-09b06a153126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:35.179642 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba35b3-b5e1-421a-84b8-09b06a153126 admin admin] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:35.179926 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba35b3-b5e1-421a-84b8-09b06a153126 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:35.180199 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba35b3-b5e1-421a-84b8-09b06a153126 admin admin] PATH_INFO: `/v3/projects` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:35.180847 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba35b3-b5e1-421a-84b8-09b06a153126 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:35.183112 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba35b3-b5e1-421a-84b8-09b06a153126 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:35.266728 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1443/2874] 10.222.0.105 () {60 vars in 1158 bytes} [Wed Aug 24 05:13:35 2022] POST /identity/v3/projects => generated 378 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:13:35.271635 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999764be-1727-4c30-8fb6-d730ec92517f None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:35.284059 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999764be-1727-4c30-8fb6-d730ec92517f None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:35.348809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1db9bb-8856-40e3-863b-7c4c197ad440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:35.349783 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1db9bb-8856-40e3-863b-7c4c197ad440 admin admin] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:35.350165 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1db9bb-8856-40e3-863b-7c4c197ad440 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:35.350442 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1db9bb-8856-40e3-863b-7c4c197ad440 admin admin] PATH_INFO: `/v3/users` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:35.351199 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1db9bb-8856-40e3-863b-7c4c197ad440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:35.353641 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1db9bb-8856-40e3-863b-7c4c197ad440 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:35.475430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1432/2875] 10.222.0.105 () {60 vars in 1149 bytes} [Wed Aug 24 05:13:35 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:13:35.477185 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba35b3-b5e1-421a-84b8-09b06a153126 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:35.492022 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba35b3-b5e1-421a-84b8-09b06a153126 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:35.499869 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7193c-2461-4ce4-bcb8-b3aa04b132b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:35.500874 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7193c-2461-4ce4-bcb8-b3aa04b132b6 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:35.501326 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7193c-2461-4ce4-bcb8-b3aa04b132b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:35.501680 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7193c-2461-4ce4-bcb8-b3aa04b132b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:35.502381 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7193c-2461-4ce4-bcb8-b3aa04b132b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:35.505672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7193c-2461-4ce4-bcb8-b3aa04b132b6 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:35.523116 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1444/2876] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:13:35 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:13:35.525325 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1db9bb-8856-40e3-863b-7c4c197ad440 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:35.536975 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1db9bb-8856-40e3-863b-7c4c197ad440 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:35.542320 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a784e5-1fb8-4c49-a5e7-bd89b2fab084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:35.543130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a784e5-1fb8-4c49-a5e7-bd89b2fab084 admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:35.543397 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a784e5-1fb8-4c49-a5e7-bd89b2fab084 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:35.543690 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a784e5-1fb8-4c49-a5e7-bd89b2fab084 admin admin] PATH_INFO: `/v3/projects/79860f8bcfbd43818f357214ad5e47ec/users/f9dd2afaeda34cc28f258c0fd5a698c5/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:35.560301 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a784e5-1fb8-4c49-a5e7-bd89b2fab084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79860f8bcfbd43818f357214ad5e47ec, user_id=f9dd2afaeda34cc28f258c0fd5a698c5, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:35.563351 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a784e5-1fb8-4c49-a5e7-bd89b2fab084 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:35.688485 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1433/2877] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:13:35 2022] PUT /identity/v3/projects/79860f8bcfbd43818f357214ad5e47ec/users/f9dd2afaeda34cc28f258c0fd5a698c5/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:13:35.693877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7193c-2461-4ce4-bcb8-b3aa04b132b6 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:35.724081 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7193c-2461-4ce4-bcb8-b3aa04b132b6 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:35.802743 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951609c9-4340-4f43-84b8-78bc98f16f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:35.803480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951609c9-4340-4f43-84b8-78bc98f16f02 admin admin] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:35.803797 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951609c9-4340-4f43-84b8-78bc98f16f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:35.804060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951609c9-4340-4f43-84b8-78bc98f16f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:35.804664 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951609c9-4340-4f43-84b8-78bc98f16f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:35.807252 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951609c9-4340-4f43-84b8-78bc98f16f02 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:35.818884 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1445/2878] 10.222.0.105 () {58 vars in 1127 bytes} [Wed Aug 24 05:13:35 2022] GET /identity/v3/roles => generated 1427 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:13:35.825001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a784e5-1fb8-4c49-a5e7-bd89b2fab084 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:35.837164 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a784e5-1fb8-4c49-a5e7-bd89b2fab084 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:35.842423 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253431c4-77ba-4a52-a64e-c31f3e8a391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:35.843107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253431c4-77ba-4a52-a64e-c31f3e8a391a admin admin] REQUEST_METHOD: `PUT` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:35.843418 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253431c4-77ba-4a52-a64e-c31f3e8a391a admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:35.843692 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253431c4-77ba-4a52-a64e-c31f3e8a391a admin admin] PATH_INFO: `/v3/projects/79860f8bcfbd43818f357214ad5e47ec/users/f9dd2afaeda34cc28f258c0fd5a698c5/roles/72df4feaecfe48d1a12346e9c9ede92a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:35.845666 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253431c4-77ba-4a52-a64e-c31f3e8a391a admin admin] RBAC: Authorizing `identity:create_grant(project_id=79860f8bcfbd43818f357214ad5e47ec, user_id=f9dd2afaeda34cc28f258c0fd5a698c5, role_id=72df4feaecfe48d1a12346e9c9ede92a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:35.853381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253431c4-77ba-4a52-a64e-c31f3e8a391a admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:35.865181 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1434/2879] 10.222.0.105 () {60 vars in 1488 bytes} [Wed Aug 24 05:13:35 2022] PUT /identity/v3/projects/79860f8bcfbd43818f357214ad5e47ec/users/f9dd2afaeda34cc28f258c0fd5a698c5/roles/72df4feaecfe48d1a12346e9c9ede92a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:13:38.012895 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951609c9-4340-4f43-84b8-78bc98f16f02 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:38.015957 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-951609c9-4340-4f43-84b8-78bc98f16f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:38.033938 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951609c9-4340-4f43-84b8-78bc98f16f02 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:38.126561 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:38.127339 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:38.127618 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:38.129063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] PATH_INFO: `/v3/users/dd01c9ee077b49c6820ecc2c6ee94046` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:38.130958 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd01c9ee077b49c6820ecc2c6ee94046)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:38.133163 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:38.273356 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:13:38.353148 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd01c9ee077b49c6820ecc2c6ee94046'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:38.353981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd01c9ee077b49c6820ecc2c6ee94046'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:38.354862 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd01c9ee077b49c6820ecc2c6ee94046'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:38.421775 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd01c9ee077b49c6820ecc2c6ee94046'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:38.436836 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd01c9ee077b49c6820ecc2c6ee94046'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:38.457636 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1446/2880] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:13:38 2022] DELETE /identity/v3/users/dd01c9ee077b49c6820ecc2c6ee94046 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:13:38.459891 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c1e42-ae88-49be-beef-91fff062a0bd None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:38.460356 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c1e42-ae88-49be-beef-91fff062a0bd None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:38.460445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c1e42-ae88-49be-beef-91fff062a0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:38.492217 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-468c1e42-ae88-49be-beef-91fff062a0bd None None] MFA Rules not processed for user `f6a4ad480e8041b39ccfa602ac02988c`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:13:38.524506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-468c1e42-ae88-49be-beef-91fff062a0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:38.610773 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1435/2881] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:13:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:13:38.620563 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:38.621981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:38.694489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6fe4c-c17c-48cc-b38d-17c7afcbefa8 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:38.699267 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf021717-168e-4b25-904f-7290dc4ec471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:38.699267 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf021717-168e-4b25-904f-7290dc4ec471 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:38.699267 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf021717-168e-4b25-904f-7290dc4ec471 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:38.704299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf021717-168e-4b25-904f-7290dc4ec471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:38.713160 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf021717-168e-4b25-904f-7290dc4ec471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:38.714693 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf021717-168e-4b25-904f-7290dc4ec471 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:38.740584 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1447/2882] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:13:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:13:39.900709 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c1e42-ae88-49be-beef-91fff062a0bd None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:39.903130 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-468c1e42-ae88-49be-beef-91fff062a0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:39.932746 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c1e42-ae88-49be-beef-91fff062a0bd None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:39.961848 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:39.962553 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:39.962823 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:39.963728 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] PATH_INFO: `/v3/projects/382c3868ca9d415c9b7157f4fdc29c30` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:39.965172 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=382c3868ca9d415c9b7157f4fdc29c30)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:39.968771 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:39.978241 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:13:40.120327 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '382c3868ca9d415c9b7157f4fdc29c30', 'request_id': 'req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:40.131596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '382c3868ca9d415c9b7157f4fdc29c30', 'request_id': 'req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:40.215196 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '382c3868ca9d415c9b7157f4fdc29c30', 'request_id': 'req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:40.237488 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] The token cache is being invalidate because project 382c3868ca9d415c9b7157f4fdc29c30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:13:40.238137 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:40.239145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:13:40.243437 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1436/2883] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:13:39 2022] DELETE /identity/v3/projects/382c3868ca9d415c9b7157f4fdc29c30 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:13:42.498225 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500079b-f366-40ba-98a1-08e12e65e4c1 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:42.498757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500079b-f366-40ba-98a1-08e12e65e4c1 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:42.498757 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500079b-f366-40ba-98a1-08e12e65e4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:42.531663 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-e500079b-f366-40ba-98a1-08e12e65e4c1 None None] MFA Rules not processed for user `f9dd2afaeda34cc28f258c0fd5a698c5`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:13:42.533006 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-e500079b-f366-40ba-98a1-08e12e65e4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:42.612077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1448/2884] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:13:42 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:13:42.629846 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:13:42.631596 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:42.709159 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9c6be1-fecc-4058-a0d1-b27ba6b28293 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:13:42.714765 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb832f-accc-4503-bb21-2b82b2ebfb20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:13:42.715476 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb832f-accc-4503-bb21-2b82b2ebfb20 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:13:42.715755 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb832f-accc-4503-bb21-2b82b2ebfb20 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:13:42.716024 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb832f-accc-4503-bb21-2b82b2ebfb20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:13:42.717757 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d0fb832f-accc-4503-bb21-2b82b2ebfb20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:13:42.787820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb832f-accc-4503-bb21-2b82b2ebfb20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:13:42.789576 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb832f-accc-4503-bb21-2b82b2ebfb20 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:13:42.800420 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1437/2885] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:13:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:13:52.608786 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=79,keystone:DELETE=5,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:13:52.795571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=1 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:14:08.880998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463c8fd-c558-4cc7-ac47-9423a94bd0e8 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:08.881559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463c8fd-c558-4cc7-ac47-9423a94bd0e8 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:08.881559 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463c8fd-c558-4cc7-ac47-9423a94bd0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:08.901539 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-4463c8fd-c558-4cc7-ac47-9423a94bd0e8 None None] MFA Rules not processed for user `11c7bb2f319e4bd2a7e03a4f77d29c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:14:08.902751 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4463c8fd-c558-4cc7-ac47-9423a94bd0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:08.909832 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1449/2886] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:14:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:14:09.330524 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb832f-accc-4503-bb21-2b82b2ebfb20 service nova] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:09.332036 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d0fb832f-accc-4503-bb21-2b82b2ebfb20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:09.392121 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb832f-accc-4503-bb21-2b82b2ebfb20 service nova] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:09.400037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae63005-5a72-4c3b-a3cb-810610a1e112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:09.400037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63005-5a72-4c3b-a3cb-810610a1e112 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:09.400037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63005-5a72-4c3b-a3cb-810610a1e112 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:09.400037 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae63005-5a72-4c3b-a3cb-810610a1e112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:09.400534 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-4ae63005-5a72-4c3b-a3cb-810610a1e112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:09.409508 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae63005-5a72-4c3b-a3cb-810610a1e112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:09.410919 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae63005-5a72-4c3b-a3cb-810610a1e112 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:09.419904 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1438/2887] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:14:09 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:14:11.323367 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463c8fd-c558-4cc7-ac47-9423a94bd0e8 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:11.324795 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4463c8fd-c558-4cc7-ac47-9423a94bd0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:11.400395 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463c8fd-c558-4cc7-ac47-9423a94bd0e8 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:11.408153 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676064e3-1277-445e-b2a0-7d2b4e31c459 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:11.408833 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676064e3-1277-445e-b2a0-7d2b4e31c459 service placement] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:11.409114 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676064e3-1277-445e-b2a0-7d2b4e31c459 service placement] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:11.409389 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676064e3-1277-445e-b2a0-7d2b4e31c459 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:11.411330 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-676064e3-1277-445e-b2a0-7d2b4e31c459 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:11.424424 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676064e3-1277-445e-b2a0-7d2b4e31c459 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:11.426060 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676064e3-1277-445e-b2a0-7d2b4e31c459 service placement] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:11.437220 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1450/2888] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:14:11 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:14:11.600084 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ce8b8-3843-43da-b316-be1e76e74024 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:11.600611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ce8b8-3843-43da-b316-be1e76e74024 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:11.600611 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ce8b8-3843-43da-b316-be1e76e74024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:11.620304 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-469ce8b8-3843-43da-b316-be1e76e74024 None None] MFA Rules not processed for user `da66e763177c47e39b3475fc42ccc8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:14:11.621608 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-469ce8b8-3843-43da-b316-be1e76e74024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:11.683082 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1439/2889] 10.222.0.105 () {58 vars in 972 bytes} [Wed Aug 24 05:14:11 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:14:11.895352 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676064e3-1277-445e-b2a0-7d2b4e31c459 service placement] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:11.902261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-676064e3-1277-445e-b2a0-7d2b4e31c459 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:11.994453 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676064e3-1277-445e-b2a0-7d2b4e31c459 service placement] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:11.999547 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff27e3b-5008-4063-91fe-6de877bdfe61 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6a894e4e6f24febbd8397fb8c4c7fb4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:12.000150 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff27e3b-5008-4063-91fe-6de877bdfe61 service swift] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:12.000423 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff27e3b-5008-4063-91fe-6de877bdfe61 service swift] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:12.000688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff27e3b-5008-4063-91fe-6de877bdfe61 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:12.010135 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff27e3b-5008-4063-91fe-6de877bdfe61 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:12.011829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff27e3b-5008-4063-91fe-6de877bdfe61 service swift] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:12.020155 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1451/2890] 10.222.0.105 () {60 vars in 1389 bytes} [Wed Aug 24 05:14:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:14:13.034894 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ce8b8-3843-43da-b316-be1e76e74024 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:13.037170 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-469ce8b8-3843-43da-b316-be1e76e74024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:13.048663 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ce8b8-3843-43da-b316-be1e76e74024 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:13.053445 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6bb3c-d98e-4f9f-bb15-22baf5533c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:13.054087 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6bb3c-d98e-4f9f-bb15-22baf5533c32 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:13.054365 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6bb3c-d98e-4f9f-bb15-22baf5533c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:13.054633 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6bb3c-d98e-4f9f-bb15-22baf5533c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:13.056498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-d3b6bb3c-d98e-4f9f-bb15-22baf5533c32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:13.132101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6bb3c-d98e-4f9f-bb15-22baf5533c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:13.133403 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6bb3c-d98e-4f9f-bb15-22baf5533c32 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:13.143128 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1440/2891] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:14:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:14:16.677981 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff27e3b-5008-4063-91fe-6de877bdfe61 service swift] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:16.692283 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff27e3b-5008-4063-91fe-6de877bdfe61 service swift] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:16.699573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca99bdd-cf23-485b-9780-c9eef04fac1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:16.700366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99bdd-cf23-485b-9780-c9eef04fac1a service placement] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:16.701041 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99bdd-cf23-485b-9780-c9eef04fac1a service placement] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:16.701450 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca99bdd-cf23-485b-9780-c9eef04fac1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:16.704084 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-1ca99bdd-cf23-485b-9780-c9eef04fac1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:16.717369 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca99bdd-cf23-485b-9780-c9eef04fac1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:16.719147 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca99bdd-cf23-485b-9780-c9eef04fac1a service placement] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:16.730956 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1452/2892] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:14:16 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:14:21.634936 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6bb3c-d98e-4f9f-bb15-22baf5533c32 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:21.645104 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6bb3c-d98e-4f9f-bb15-22baf5533c32 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:21.651453 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2f3ac-15c4-4c15-a196-d83d8db4baf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:21.652052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2f3ac-15c4-4c15-a196-d83d8db4baf6 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:21.652424 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2f3ac-15c4-4c15-a196-d83d8db4baf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:21.652713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2f3ac-15c4-4c15-a196-d83d8db4baf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:21.654817 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-05a2f3ac-15c4-4c15-a196-d83d8db4baf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:21.673030 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2f3ac-15c4-4c15-a196-d83d8db4baf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:21.674473 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2f3ac-15c4-4c15-a196-d83d8db4baf6 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:21.687223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1441/2893] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:14:21 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:14:22.878517 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca99bdd-cf23-485b-9780-c9eef04fac1a service placement] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:22.889047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca99bdd-cf23-485b-9780-c9eef04fac1a service placement] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:22.892752 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be4464-abcc-47c4-a533-11150787dc49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:22.893343 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be4464-abcc-47c4-a533-11150787dc49 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:22.893616 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be4464-abcc-47c4-a533-11150787dc49 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:22.893885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be4464-abcc-47c4-a533-11150787dc49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:22.895661 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-83be4464-abcc-47c4-a533-11150787dc49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:22.914388 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be4464-abcc-47c4-a533-11150787dc49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:22.916378 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2f3ac-15c4-4c15-a196-d83d8db4baf6 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:22.917749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be4464-abcc-47c4-a533-11150787dc49 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:22.923616 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-05a2f3ac-15c4-4c15-a196-d83d8db4baf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:22.934855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1453/2894] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:14:22 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:14:22.936018 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2f3ac-15c4-4c15-a196-d83d8db4baf6 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:22.941456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:22.942093 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:22.942575 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:22.942915 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] PATH_INFO: `/v3/users/dfc1475f560d4151a0c569433544f36a` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:22.944809 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfc1475f560d4151a0c569433544f36a)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:22.946923 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:22.985805 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:14:23.151654 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfc1475f560d4151a0c569433544f36a'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:23.152401 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfc1475f560d4151a0c569433544f36a'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:23.153041 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfc1475f560d4151a0c569433544f36a'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:23.205209 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfc1475f560d4151a0c569433544f36a'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:23.218899 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfc1475f560d4151a0c569433544f36a'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:23.231853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1442/2895] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:14:22 2022] DELETE /identity/v3/users/dfc1475f560d4151a0c569433544f36a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:14:23.504354 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be4464-abcc-47c4-a533-11150787dc49 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:23.516463 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be4464-abcc-47c4-a533-11150787dc49 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:23.553261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=100 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:14:23.577380 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:23.578005 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:23.578279 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:23.578546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] PATH_INFO: `/v3/projects/67112706d4df4183bc338732b237df6e` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:23.579699 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67112706d4df4183bc338732b237df6e)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:23.581577 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:23.702675 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67112706d4df4183bc338732b237df6e', 'request_id': 'req-c87efee6-9b52-45e0-8a07-2e6241fadf56'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:23.710681 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67112706d4df4183bc338732b237df6e', 'request_id': 'req-c87efee6-9b52-45e0-8a07-2e6241fadf56'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:23.717077 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67112706d4df4183bc338732b237df6e', 'request_id': 'req-c87efee6-9b52-45e0-8a07-2e6241fadf56'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:23.791334 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] The token cache is being invalidate because project 67112706d4df4183bc338732b237df6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:14:23.791892 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:23.792575 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:23.794393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1454/2896] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:14:23 2022] DELETE /identity/v3/projects/67112706d4df4183bc338732b237df6e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:14:23.798396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:23.800115 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:23.814698 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fc09d-b7cd-4a0b-980a-318a658e8118 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:23.877231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:23.877886 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:23.878163 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:23.878433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] PATH_INFO: `/v3/users/4ae6c4dd34e643999d652056fd911f97` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:23.891038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ae6c4dd34e643999d652056fd911f97)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:23.891038 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:24.044571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ae6c4dd34e643999d652056fd911f97'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:24.045390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ae6c4dd34e643999d652056fd911f97'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:24.046006 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ae6c4dd34e643999d652056fd911f97'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:24.132572 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ae6c4dd34e643999d652056fd911f97'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:24.149696 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ae6c4dd34e643999d652056fd911f97'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:24.164408 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1443/2897] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:14:23 2022] DELETE /identity/v3/users/4ae6c4dd34e643999d652056fd911f97 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:14:24.442637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:24.444500 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:24.458090 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87efee6-9b52-45e0-8a07-2e6241fadf56 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:24.520885 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:24.522127 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:24.522411 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:24.522686 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] PATH_INFO: `/v3/projects/08adf9792f4f45db9d1c98108116ec31` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:24.523877 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=08adf9792f4f45db9d1c98108116ec31)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:24.525840 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:24.640425 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08adf9792f4f45db9d1c98108116ec31', 'request_id': 'req-c6f8daed-4e7c-426b-8cc2-52c971d193ff'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:24.651650 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08adf9792f4f45db9d1c98108116ec31', 'request_id': 'req-c6f8daed-4e7c-426b-8cc2-52c971d193ff'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:24.657393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08adf9792f4f45db9d1c98108116ec31', 'request_id': 'req-c6f8daed-4e7c-426b-8cc2-52c971d193ff'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:24.664442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:14:24.747284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] The token cache is being invalidate because project 08adf9792f4f45db9d1c98108116ec31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82575) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:14:24.747576 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:24.748210 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:14:24.749711 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1455/2898] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:14:24 2022] DELETE /identity/v3/projects/08adf9792f4f45db9d1c98108116ec31 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:14:34.159433 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=73,keystone:DELETE=16,keystone:INSERT=2 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:14:34.862288 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=2 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:14:36.330620 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:14:36.332082 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:36.390628 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ea700-8cf6-42f4-aade-86f52be1648c admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:14:36.441842 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a301d-a51e-4bdf-ab40-c9350132b926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:14:36.442462 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a301d-a51e-4bdf-ab40-c9350132b926 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:14:36.442739 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a301d-a51e-4bdf-ab40-c9350132b926 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:14:36.443007 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a301d-a51e-4bdf-ab40-c9350132b926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:14:36.444844 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7a4a301d-a51e-4bdf-ab40-c9350132b926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:14:36.513804 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a301d-a51e-4bdf-ab40-c9350132b926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:14:36.515284 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a301d-a51e-4bdf-ab40-c9350132b926 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:14:36.525194 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1444/2899] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:14:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:14:46.520774 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=70 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:15:40.437106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:15:40.438538 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:15:40.451428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8daed-4e7c-426b-8cc2-52c971d193ff admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:15:40.455117 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09731f88-5caa-4bca-914a-17d25e23e20e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:15:40.455724 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09731f88-5caa-4bca-914a-17d25e23e20e service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:15:40.455998 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09731f88-5caa-4bca-914a-17d25e23e20e service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:15:40.456264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09731f88-5caa-4bca-914a-17d25e23e20e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:15:40.457971 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-09731f88-5caa-4bca-914a-17d25e23e20e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:15:40.469749 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09731f88-5caa-4bca-914a-17d25e23e20e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:15:40.471227 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09731f88-5caa-4bca-914a-17d25e23e20e service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:15:40.480162 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1456/2900] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:15:40 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:15:50.476274 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=6 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:16:57.592577 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a301d-a51e-4bdf-ab40-c9350132b926 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:16:57.594615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-7a4a301d-a51e-4bdf-ab40-c9350132b926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:16:57.692791 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a301d-a51e-4bdf-ab40-c9350132b926 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:16:57.709428 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4c7c1-1a6a-454c-9adf-505154b86184 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:16:57.712615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4c7c1-1a6a-454c-9adf-505154b86184 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:16:57.712615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4c7c1-1a6a-454c-9adf-505154b86184 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:16:57.712615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4c7c1-1a6a-454c-9adf-505154b86184 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:16:57.712615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-dda4c7c1-1a6a-454c-9adf-505154b86184 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:16:57.730969 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4c7c1-1a6a-454c-9adf-505154b86184 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:16:57.732772 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4c7c1-1a6a-454c-9adf-505154b86184 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:16:57.751236 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1445/2901] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:16:57 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:16:59.493976 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2507553-3794-4f20-b034-949ea8580ea7 None None] REQUEST_METHOD: `POST` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:16:59.494502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2507553-3794-4f20-b034-949ea8580ea7 None None] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:16:59.494502 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2507553-3794-4f20-b034-949ea8580ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:16:59.514366 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.auth.core [None req-c2507553-3794-4f20-b034-949ea8580ea7 None None] MFA Rules not processed for user `11c7bb2f319e4bd2a7e03a4f77d29c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=82575) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:16:59.515612 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-c2507553-3794-4f20-b034-949ea8580ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:16:59.573211 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1457/2902] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:16:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:17:07.741000 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=34 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:17:09.565834 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=25 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:17:16.020533 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4c7c1-1a6a-454c-9adf-505154b86184 service nova] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:17:16.023398 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-dda4c7c1-1a6a-454c-9adf-505154b86184 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:17:16.036631 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4c7c1-1a6a-454c-9adf-505154b86184 service nova] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:17:16.042350 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21254be1-3023-4b35-a0c7-0d91fb08fcf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:17:16.042979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21254be1-3023-4b35-a0c7-0d91fb08fcf0 service nova] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:17:16.043289 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21254be1-3023-4b35-a0c7-0d91fb08fcf0 service nova] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:17:16.043560 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21254be1-3023-4b35-a0c7-0d91fb08fcf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:17:16.045319 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-21254be1-3023-4b35-a0c7-0d91fb08fcf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:17:16.057998 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21254be1-3023-4b35-a0c7-0d91fb08fcf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:17:16.059584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21254be1-3023-4b35-a0c7-0d91fb08fcf0 service nova] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:17:16.070853 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1446/2903] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:17:16 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:17:21.499417 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2507553-3794-4f20-b034-949ea8580ea7 None None] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:17:21.513235 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2507553-3794-4f20-b034-949ea8580ea7 None None] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:17:21.519430 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa971b-1fd1-4d51-8893-4b43d25fd735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:17:21.520146 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa971b-1fd1-4d51-8893-4b43d25fd735 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:17:21.520487 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa971b-1fd1-4d51-8893-4b43d25fd735 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:17:21.520829 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa971b-1fd1-4d51-8893-4b43d25fd735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:17:21.522596 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f6fa971b-1fd1-4d51-8893-4b43d25fd735 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:17:21.536182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa971b-1fd1-4d51-8893-4b43d25fd735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:17:21.537758 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fa971b-1fd1-4d51-8893-4b43d25fd735 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:17:21.550792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1458/2904] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:17:21 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:17:26.066223 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=6 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:17:31.544393 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=6 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:17:46.902222 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21254be1-3023-4b35-a0c7-0d91fb08fcf0 service nova] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:17:46.903806 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-21254be1-3023-4b35-a0c7-0d91fb08fcf0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:17:47.047245 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21254be1-3023-4b35-a0c7-0d91fb08fcf0 service nova] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:17:47.053615 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f6978-a9d3-4994-8b4b-3d0856844755 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:17:47.054441 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f6978-a9d3-4994-8b4b-3d0856844755 service placement] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:17:47.054829 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f6978-a9d3-4994-8b4b-3d0856844755 service placement] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:17:47.055145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f6978-a9d3-4994-8b4b-3d0856844755 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:17:47.057184 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-189f6978-a9d3-4994-8b4b-3d0856844755 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:17:47.069820 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f6978-a9d3-4994-8b4b-3d0856844755 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:17:47.071434 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f6978-a9d3-4994-8b4b-3d0856844755 service placement] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:17:47.081530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1447/2905] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:17:46 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:17:57.076983 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=63 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:19:10.342703 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa971b-1fd1-4d51-8893-4b43d25fd735 service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:19:10.344168 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-f6fa971b-1fd1-4d51-8893-4b43d25fd735 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:19:10.450119 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fa971b-1fd1-4d51-8893-4b43d25fd735 service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:19:10.453715 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa864936-1081-404d-bfb9-36ea3f2538a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:19:10.454310 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa864936-1081-404d-bfb9-36ea3f2538a7 service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:19:10.454586 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa864936-1081-404d-bfb9-36ea3f2538a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:19:10.454857 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa864936-1081-404d-bfb9-36ea3f2538a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:19:10.456601 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fa864936-1081-404d-bfb9-36ea3f2538a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:19:10.468029 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa864936-1081-404d-bfb9-36ea3f2538a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:19:10.469428 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa864936-1081-404d-bfb9-36ea3f2538a7 service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:19:10.478208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1459/2906] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:19:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:19:20.474322 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=48 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:19:23.006001 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb316f-977e-467c-b56b-53639bfaebd1 None None] REQUEST_METHOD: `POST` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:19:23.006571 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb316f-977e-467c-b56b-53639bfaebd1 None None] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:19:23.006713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb316f-977e-467c-b56b-53639bfaebd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:19:23.028798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.auth.core [None req-b5bb316f-977e-467c-b56b-53639bfaebd1 None None] MFA Rules not processed for user `11c7bb2f319e4bd2a7e03a4f77d29c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=82574) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:19:23.030200 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b5bb316f-977e-467c-b56b-53639bfaebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:19:23.039694 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1448/2907] 10.222.0.105 () {58 vars in 963 bytes} [Wed Aug 24 05:19:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:19:23.084094 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa864936-1081-404d-bfb9-36ea3f2538a7 service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:19:23.085697 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-fa864936-1081-404d-bfb9-36ea3f2538a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:19:23.098442 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa864936-1081-404d-bfb9-36ea3f2538a7 service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:19:23.102218 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518be9-b20f-4b17-97b2-b7abbf06bead service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29af87e1e05b4880b9ec96865994b635', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:19:23.102838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518be9-b20f-4b17-97b2-b7abbf06bead service cinder] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:19:23.103143 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518be9-b20f-4b17-97b2-b7abbf06bead service cinder] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:19:23.103416 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518be9-b20f-4b17-97b2-b7abbf06bead service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:19:23.105261 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-71518be9-b20f-4b17-97b2-b7abbf06bead service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:19:23.139342 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518be9-b20f-4b17-97b2-b7abbf06bead service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:19:23.141802 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71518be9-b20f-4b17-97b2-b7abbf06bead service cinder] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:19:23.164530 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1460/2908] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:19:23 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:19:23.455602 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb316f-977e-467c-b56b-53639bfaebd1 None None] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:19:23.456989 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-b5bb316f-977e-467c-b56b-53639bfaebd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:19:23.491484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb316f-977e-467c-b56b-53639bfaebd1 None None] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:19:23.495116 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2f17d-2a5d-49aa-8c40-388eac69fa9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:19:23.495713 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2f17d-2a5d-49aa-8c40-388eac69fa9b service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:19:23.495990 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2f17d-2a5d-49aa-8c40-388eac69fa9b service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:19:23.496262 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2f17d-2a5d-49aa-8c40-388eac69fa9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:19:23.497981 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-ebf2f17d-2a5d-49aa-8c40-388eac69fa9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:19:23.508498 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2f17d-2a5d-49aa-8c40-388eac69fa9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:19:23.509897 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf2f17d-2a5d-49aa-8c40-388eac69fa9b service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:19:23.521134 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1449/2909] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:19:23 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:19:25.202851 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518be9-b20f-4b17-97b2-b7abbf06bead service cinder] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:19:25.212600 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71518be9-b20f-4b17-97b2-b7abbf06bead service cinder] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:19:25.216301 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244de2e2-8aaf-4893-807f-763382d00e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:19:25.216922 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244de2e2-8aaf-4893-807f-763382d00e16 service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:19:25.217193 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244de2e2-8aaf-4893-807f-763382d00e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:19:25.217489 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244de2e2-8aaf-4893-807f-763382d00e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:19:25.219705 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-244de2e2-8aaf-4893-807f-763382d00e16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:19:25.236605 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244de2e2-8aaf-4893-807f-763382d00e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:19:25.238294 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244de2e2-8aaf-4893-807f-763382d00e16 service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:19:25.251241 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1461/2910] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:19:25 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:19:26.201490 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2f17d-2a5d-49aa-8c40-388eac69fa9b service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:19:26.213954 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2f17d-2a5d-49aa-8c40-388eac69fa9b service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:19:26.219993 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497eae0-9745-4dc3-bae8-9d4d0b884afc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:19:26.220584 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497eae0-9745-4dc3-bae8-9d4d0b884afc service placement] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:19:26.220862 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497eae0-9745-4dc3-bae8-9d4d0b884afc service placement] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:19:26.221132 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497eae0-9745-4dc3-bae8-9d4d0b884afc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:19:26.222859 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-2497eae0-9745-4dc3-bae8-9d4d0b884afc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:19:26.235908 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497eae0-9745-4dc3-bae8-9d4d0b884afc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:19:26.237482 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497eae0-9745-4dc3-bae8-9d4d0b884afc service placement] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:19:26.248381 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1450/2911] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:19:26 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:19:35.245347 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=24 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:19:36.244108 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=24 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:19:47.647328 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244de2e2-8aaf-4893-807f-763382d00e16 service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:19:47.656540 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244de2e2-8aaf-4893-807f-763382d00e16 service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:19:47.661065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a53869-b8f2-4304-a89f-b6b91f93295a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:19:47.661659 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a53869-b8f2-4304-a89f-b6b91f93295a service neutron] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:19:47.661939 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a53869-b8f2-4304-a89f-b6b91f93295a service neutron] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:19:47.662208 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a53869-b8f2-4304-a89f-b6b91f93295a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:19:47.672546 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a53869-b8f2-4304-a89f-b6b91f93295a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:19:47.673961 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a53869-b8f2-4304-a89f-b6b91f93295a service neutron] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:19:47.684017 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1462/2912] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:19:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:19:57.679672 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=6 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:20:46.402761 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497eae0-9745-4dc3-bae8-9d4d0b884afc service placement] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:20:46.412687 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497eae0-9745-4dc3-bae8-9d4d0b884afc service placement] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:20:46.416390 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481b496-822a-4cda-9702-b5b5b4ae7149 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:20:46.416986 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481b496-822a-4cda-9702-b5b5b4ae7149 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:20:46.417263 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481b496-822a-4cda-9702-b5b5b4ae7149 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:20:46.417530 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481b496-822a-4cda-9702-b5b5b4ae7149 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:20:46.424855 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481b496-822a-4cda-9702-b5b5b4ae7149 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:20:46.426298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481b496-822a-4cda-9702-b5b5b4ae7149 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:20:46.434484 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1451/2913] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:20:46 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:20:56.430960 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=6 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:22:01.716850 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a53869-b8f2-4304-a89f-b6b91f93295a service neutron] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:01.729792 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a53869-b8f2-4304-a89f-b6b91f93295a service neutron] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:01.735676 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49109ceb-1018-49fc-a2fb-819368002725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c822c8f23b74bae84769046fdb85db4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:01.736362 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49109ceb-1018-49fc-a2fb-819368002725 service nova] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:01.736648 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49109ceb-1018-49fc-a2fb-819368002725 service nova] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:01.736933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49109ceb-1018-49fc-a2fb-819368002725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:01.747065 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49109ceb-1018-49fc-a2fb-819368002725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:01.748713 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49109ceb-1018-49fc-a2fb-819368002725 service nova] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:01.760480 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1463/2914] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:22:01 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:22:06.185206 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481b496-822a-4cda-9702-b5b5b4ae7149 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:06.195207 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481b496-822a-4cda-9702-b5b5b4ae7149 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:06.199116 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc84cb8-0c2d-4572-948f-d7688bfa4f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44d2b4015ac74b8b8d0a674419f34cf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:06.199798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc84cb8-0c2d-4572-948f-d7688bfa4f20 service neutron] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:06.200072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc84cb8-0c2d-4572-948f-d7688bfa4f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:06.200338 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc84cb8-0c2d-4572-948f-d7688bfa4f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:06.203430 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-acc84cb8-0c2d-4572-948f-d7688bfa4f20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:22:06.294107 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc84cb8-0c2d-4572-948f-d7688bfa4f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:06.295468 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc84cb8-0c2d-4572-948f-d7688bfa4f20 service neutron] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:06.316145 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1452/2915] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:22:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:22:09.816176 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49109ceb-1018-49fc-a2fb-819368002725 service nova] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:09.817694 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-49109ceb-1018-49fc-a2fb-819368002725 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:22:09.832549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49109ceb-1018-49fc-a2fb-819368002725 service nova] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:09.838045 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae18e6f-6a99-4730-83df-aa4ab8ffde15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:09.838652 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae18e6f-6a99-4730-83df-aa4ab8ffde15 service placement] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:09.838933 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae18e6f-6a99-4730-83df-aa4ab8ffde15 service placement] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:09.839263 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae18e6f-6a99-4730-83df-aa4ab8ffde15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:09.840975 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4ae18e6f-6a99-4730-83df-aa4ab8ffde15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:22:09.853994 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae18e6f-6a99-4730-83df-aa4ab8ffde15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:09.855671 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae18e6f-6a99-4730-83df-aa4ab8ffde15 service placement] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:09.866844 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1464/2916] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:22:09 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:22:13.335119 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc84cb8-0c2d-4572-948f-d7688bfa4f20 service neutron] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:13.348798 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc84cb8-0c2d-4572-948f-d7688bfa4f20 service neutron] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:13.353825 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab0c76-8ab4-4cba-9318-bff9b80607e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:13.354472 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab0c76-8ab4-4cba-9318-bff9b80607e7 service placement] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:13.354778 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab0c76-8ab4-4cba-9318-bff9b80607e7 service placement] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:13.355090 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab0c76-8ab4-4cba-9318-bff9b80607e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:13.356827 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-1fab0c76-8ab4-4cba-9318-bff9b80607e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:22:13.369753 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab0c76-8ab4-4cba-9318-bff9b80607e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:13.371333 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fab0c76-8ab4-4cba-9318-bff9b80607e7 service placement] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:13.381639 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1453/2917] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:22:13 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:22:13.458182 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae18e6f-6a99-4730-83df-aa4ab8ffde15 service placement] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:13.460549 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-4ae18e6f-6a99-4730-83df-aa4ab8ffde15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:22:13.476085 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae18e6f-6a99-4730-83df-aa4ab8ffde15 service placement] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:13.480665 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:13.481284 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:13.481564 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:13.481838 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] PATH_INFO: `/v3/users/11c7bb2f319e4bd2a7e03a4f77d29c4f` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:13.482960 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c7bb2f319e4bd2a7e03a4f77d29c4f)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:13.485161 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:13.643932 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c7bb2f319e4bd2a7e03a4f77d29c4f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:13.644529 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c7bb2f319e4bd2a7e03a4f77d29c4f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:13.645149 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c7bb2f319e4bd2a7e03a4f77d29c4f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:13.686221 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c7bb2f319e4bd2a7e03a4f77d29c4f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:13.699264 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c7bb2f319e4bd2a7e03a4f77d29c4f'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:13.711656 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1465/2918] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:22:13 2022] DELETE /identity/v3/users/11c7bb2f319e4bd2a7e03a4f77d29c4f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:22:13.945298 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab0c76-8ab4-4cba-9318-bff9b80607e7 service placement] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:13.957979 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab0c76-8ab4-4cba-9318-bff9b80607e7 service placement] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:14.013302 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:14.013928 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:14.014204 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:14.014475 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] PATH_INFO: `/v3/projects/2a010cb56074448a979b3b7d944a92d6` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:14.016174 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a010cb56074448a979b3b7d944a92d6)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:14.018052 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:14.127410 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a010cb56074448a979b3b7d944a92d6', 'request_id': 'req-1b437d98-3dc5-40c2-93d2-298cf24413f9'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:14.134101 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a010cb56074448a979b3b7d944a92d6', 'request_id': 'req-1b437d98-3dc5-40c2-93d2-298cf24413f9'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:14.136011 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:22:14.194183 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a010cb56074448a979b3b7d944a92d6', 'request_id': 'req-1b437d98-3dc5-40c2-93d2-298cf24413f9'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:14.207581 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] The token cache is being invalidate because project 2a010cb56074448a979b3b7d944a92d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:22:14.207870 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:14.208590 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:14.210291 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1454/2919] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:22:13 2022] DELETE /identity/v3/projects/2a010cb56074448a979b3b7d944a92d6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:22:14.214197 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:14.215608 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:22:14.229186 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb541b-d457-45c9-a6bd-e55f6dcf3e9d admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:14.286063 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:14.286688 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:14.286968 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:14.287268 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] PATH_INFO: `/v3/users/f9dd2afaeda34cc28f258c0fd5a698c5` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:14.288400 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9dd2afaeda34cc28f258c0fd5a698c5)` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:14.290299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:14.470335 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9dd2afaeda34cc28f258c0fd5a698c5'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:14.471047 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9dd2afaeda34cc28f258c0fd5a698c5'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:14.471573 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9dd2afaeda34cc28f258c0fd5a698c5'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:14.473265 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:22:14.542368 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9dd2afaeda34cc28f258c0fd5a698c5'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:14.555271 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.notifications [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9dd2afaeda34cc28f258c0fd5a698c5'} {{(pid=82575) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:14.568464 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1466/2920] 10.222.0.105 () {58 vars in 1229 bytes} [Wed Aug 24 05:22:14 2022] DELETE /identity/v3/users/f9dd2afaeda34cc28f258c0fd5a698c5 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:22:14.814738 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:14.816456 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:22:14.830563 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b437d98-3dc5-40c2-93d2-298cf24413f9 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:14.888518 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f6a4ad480e8041b39ccfa602ac02988c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e02e910be5274b36b8f63c2185a111d4', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:14.889153 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:14.889429 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:14.889701 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] PATH_INFO: `/v3/projects/79860f8bcfbd43818f357214ad5e47ec` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:14.891522 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79860f8bcfbd43818f357214ad5e47ec)` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:14.893724 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:15.001363 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79860f8bcfbd43818f357214ad5e47ec', 'request_id': 'req-43efb0fe-44c7-4bde-a7a0-ffae229b1005'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:15.007570 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79860f8bcfbd43818f357214ad5e47ec', 'request_id': 'req-43efb0fe-44c7-4bde-a7a0-ffae229b1005'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:15.086506 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79860f8bcfbd43818f357214ad5e47ec', 'request_id': 'req-43efb0fe-44c7-4bde-a7a0-ffae229b1005'} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:15.097396 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] The token cache is being invalidate because project 79860f8bcfbd43818f357214ad5e47ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=82574) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:22:15.097681 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:15.098310 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.notifications [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=82574) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:22:15.099816 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1455/2921] 10.222.0.105 () {58 vars in 1238 bytes} [Wed Aug 24 05:22:14 2022] DELETE /identity/v3/projects/79860f8bcfbd43818f357214ad5e47ec => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:22:24.610995 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=12 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:22:25.094258 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG dbcounter [-] [82574] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=55 {{(pid=82574) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:22:51.647883 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] Authenticating user token {{(pid=82575) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:22:51.649299 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:22:51.720355 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a84e3a-7dfa-4145-a7d3-85e77b72bfe2 admin admin] Validating token access rules against request {{(pid=82575) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:22:51.778855 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2839d2-e2ea-46dd-95e8-f7b4b30bdf91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82575) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:22:51.780056 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2839d2-e2ea-46dd-95e8-f7b4b30bdf91 service placement] REQUEST_METHOD: `GET` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:22:51.780337 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2839d2-e2ea-46dd-95e8-f7b4b30bdf91 service placement] SCRIPT_NAME: `/identity` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:22:51.780637 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2839d2-e2ea-46dd-95e8-f7b4b30bdf91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82575) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:22:51.782422 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.fernet_utils [None req-3b2839d2-e2ea-46dd-95e8-f7b4b30bdf91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82575) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:22:51.793762 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2839d2-e2ea-46dd-95e8-f7b4b30bdf91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:22:51.796058 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2839d2-e2ea-46dd-95e8-f7b4b30bdf91 service placement] RBAC: Authorization granted {{(pid=82575) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:22:51.806106 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: [pid: 82575|app: 0|req: 1467/2922] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:22:51 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:23:01.801720 dv-cinder-854265-1-iscsi devstack@keystone.service[82575]: DEBUG dbcounter [-] [82575] Writing DB stats keystone:SELECT=58 {{(pid=82575) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:27:54.606072 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] Authenticating user token {{(pid=82574) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:27:54.607547 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.fernet_utils [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=82574) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:27:54.667695 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efb0fe-44c7-4bde-a7a0-ffae229b1005 admin admin] Validating token access rules against request {{(pid=82574) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:27:54.672951 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d119ce-5f79-415c-b9dd-e180c5f22927 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ad8825c294534a7c88877b48b5f8f07f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd6a64369509349b4b65a772fb6057599', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=82574) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:27:54.673564 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d119ce-5f79-415c-b9dd-e180c5f22927 service placement] REQUEST_METHOD: `GET` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:27:54.673849 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d119ce-5f79-415c-b9dd-e180c5f22927 service placement] SCRIPT_NAME: `/identity` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:27:54.674120 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d119ce-5f79-415c-b9dd-e180c5f22927 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=82574) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:27:54.683651 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d119ce-5f79-415c-b9dd-e180c5f22927 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:27:54.685231 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d119ce-5f79-415c-b9dd-e180c5f22927 service placement] RBAC: Authorization granted {{(pid=82574) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:27:54.695386 dv-cinder-854265-1-iscsi devstack@keystone.service[82574]: [pid: 82574|app: 0|req: 1456/2923] 10.222.0.105 () {60 vars in 1370 bytes} [Wed Aug 24 05:27:54 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)